-- Logs begin at Tue 2022-08-23 05:21:05 UTC, end at Tue 2022-08-23 06:14:42 UTC. -- Aug 23 05:58:32.197945 dv-cinder-854070-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 23 05:58:32.208278 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 23 05:58:32.210143 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 23 05:58:32 2022] *** Aug 23 05:58:32.210143 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 23 05:58:32.210143 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 23 05:58:32.210143 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: nodename: dv-cinder-854070-2-iscsi Aug 23 05:58:32.210143 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: machine: x86_64 Aug 23 05:58:32.210143 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: clock source: unix Aug 23 05:58:32.210143 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: pcre jit disabled Aug 23 05:58:32.210143 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: detected number of CPU cores: 4 Aug 23 05:58:32.210143 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: current working directory: / Aug 23 05:58:32.210143 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: detected binary path: /usr/bin/uwsgi-core Aug 23 05:58:32.210714 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: your processes number limit is 47680 Aug 23 05:58:32.210714 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: your memory page size is 4096 bytes Aug 23 05:58:32.210714 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: detected max file descriptor number: 2048 Aug 23 05:58:32.210714 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: lock engine: pthread robust mutexes Aug 23 05:58:32.210714 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: thunder lock: enabled Aug 23 05:58:32.210714 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 23 05:58:32.210714 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 23 05:58:32.249377 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: Python main interpreter initialized at 0x559076dc39c0 Aug 23 05:58:32.249377 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: python threads support enabled Aug 23 05:58:32.249377 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: your server socket listen backlog is limited to 100 connections Aug 23 05:58:32.249377 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: your mercy for graceful operations on workers is 90 seconds Aug 23 05:58:32.249743 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: mapped 403077 bytes (393 KB) for 2 cores Aug 23 05:58:32.249818 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: *** Operational MODE: preforking *** Aug 23 05:58:32.250057 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: *** uWSGI is running in multiple interpreter mode *** Aug 23 05:58:32.250057 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: spawned uWSGI master process (pid: 81735) Aug 23 05:58:32.250252 dv-cinder-854070-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 23 05:58:32.251431 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: spawned uWSGI worker 1 (pid: 81746, cores: 1) Aug 23 05:58:32.251431 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: spawned uWSGI worker 2 (pid: 81747, cores: 1) Aug 23 05:58:32.251431 dv-cinder-854070-2-iscsi devstack@keystone.service[81735]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 23 05:58:36.561698 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 05:58:36.561698 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: from cryptography.utils import int_from_bytes Aug 23 05:58:36.562442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 05:58:36.562442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: from cryptography.utils import int_from_bytes Aug 23 05:58:36.563266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 05:58:36.563266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: from cryptography.utils import int_from_bytes Aug 23 05:58:36.563714 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 05:58:36.563714 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: from cryptography.utils import int_from_bytes Aug 23 05:58:36.614257 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 23 05:58:36.614461 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 23 05:58:36.614577 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 23 05:58:36.614668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 23 05:58:36.614746 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] command line args: [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 23 05:58:36.614822 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] command line args: [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 23 05:58:36.614900 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 23 05:58:36.615057 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ================================================================================ {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 23 05:58:36.615136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 23 05:58:36.615261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ================================================================================ {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 23 05:58:36.615386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] admin_token = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.615514 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] admin_token = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.615610 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] config_dir = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.615688 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.615758 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] config_dir = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.615871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] config_source = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.615949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.616030 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] debug = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.616114 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] config_source = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.616229 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.616477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] debug = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.616477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.616617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.616684 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.616748 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.616899 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.617005 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.617112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.617216 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.617346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.617452 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] list_limit = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.617579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] list_limit = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.617682 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] log_config_append = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.617795 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.617891 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] log_config_append = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.617991 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] log_dir = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.618087 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.618186 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] log_file = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.618293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] log_dir = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.618293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] log_file = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.618802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.618879 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.618995 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.619075 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.619192 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.619277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.619399 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.619483 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.619685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.619810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.619810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.619995 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.620077 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.620149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.620254 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.620381 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.620486 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.620569 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.620716 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.620793 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.620949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.621033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.621186 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.621266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.621424 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.621500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.621670 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.621756 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.621950 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.622181 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.622364 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] publish_errors = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.622491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.622621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.622700 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] publish_errors = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.622852 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.622947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.623021 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.623091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.623259 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.623345 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.623491 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.623579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.623723 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] standard_threads = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.623800 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.623954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.624073 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] standard_threads = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.624169 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.624243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.624354 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.624440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.624511 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] use_journal = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.624629 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.624704 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] use_json = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.624780 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] use_journal = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.624852 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] use_stderr = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.624927 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] use_json = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.624997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] use_syslog = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.625169 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] use_stderr = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.625259 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.625331 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] use_syslog = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.625455 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.625569 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 05:58:36.625691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.625828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.625947 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.626067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.626183 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.626330 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.626470 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.626689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.626823 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.627082 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.627260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.627498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.627631 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.627767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.627894 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.627976 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.628143 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.628269 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.628395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.628528 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.628665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.628788 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.628909 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.629041 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.629162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.629294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.629426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.629552 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.629684 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.629815 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.629941 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.630079 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.630205 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.630348 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.630474 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.631004 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.631132 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.631338 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.631471 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.631597 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.631729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.631815 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.631936 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.632063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.632190 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.632315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.632459 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.632584 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.632721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.632802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.632973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.633099 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.633234 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.633360 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.633493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.633622 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.633753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.633878 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.633960 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.634031 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.634108 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.634180 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.634249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.634317 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.634385 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.634453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.634576 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.634661 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.634739 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.634809 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.634884 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.634953 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.635029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.635105 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.635181 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.635251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.635324 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.635395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.635472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.635542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.635610 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.635679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.635753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.635828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.635975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.636055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.636126 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.636194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.636293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.636375 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.636531 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.636609 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.636784 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.636872 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.637013 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.637090 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.637252 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.637436 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.637521 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.637591 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.637707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.637825 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.637976 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.638073 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.638175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.638277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.638376 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.638574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.638691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.638791 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.638924 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.639073 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.639160 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.639239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.639318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.639435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.639510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.639624 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.639743 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.639830 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.639974 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.640062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.640215 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.640293 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.640453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.640542 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.640685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.640807 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.640950 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.641131 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.641215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.641340 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.641416 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.641494 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.641607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.641685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.641844 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.641924 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.642655 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.642781 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.642952 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.643135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.643324 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.643402 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.643469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.643642 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.643724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.643890 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.643992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.644113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.644217 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.644355 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.644450 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.644594 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.644716 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.644833 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.644997 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.645138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.645270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.645395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.645523 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.645651 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.645779 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.645916 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.646044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.646168 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.646304 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.646435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.646588 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.646724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.646854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.646981 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.647108 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.647235 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.647361 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.647497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.647621 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.647743 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.647871 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.648000 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.648132 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.648344 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.648477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.648598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.648725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.648853 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.649333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.649466 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.649601 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.649729 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.649856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.649983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.650118 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.650241 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.650375 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.650565 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.650690 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.650819 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.650947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.651073 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.651211 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.651339 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.651466 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.651594 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] auth.external = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.651721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.651847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.651981 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.652064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.652126 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.652268 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.652390 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.652528 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] auth.password = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.652696 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] auth.external = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.652835 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] auth.token = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.652970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.653097 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.653097 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.653339 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.653476 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.653603 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.653738 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.653865 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] auth.password = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.654002 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.654132 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] auth.token = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.654259 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.654390 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.654537 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.654661 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.654789 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] credential.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.654921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.655044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.655181 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.655316 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.655445 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.655579 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.655707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.655849 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.655977 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.656110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.656239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] credential.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.656366 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.656493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.656620 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.656746 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.656882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.657008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.657141 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.657269 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.657397 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.657533 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.657625 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.657688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.658376 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.658594 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.658732 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.658861 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.659028 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.659197 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.659327 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.659457 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.659585 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.659712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.659847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.659986 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.660113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.660240 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.660373 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.660503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.660641 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.660641 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.660883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.661010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.661137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.661264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.661399 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.661529 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.661529 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.661780 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.661988 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.662119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.662246 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.662379 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.662507 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.662648 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.662777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.662915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.662998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.663145 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.663269 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.663394 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.663528 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.663660 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.663785 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.663910 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.664034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.664158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.664281 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.664482 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.664617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.664742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.664867 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.664992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.665116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.665261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.665393 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.665517 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.665649 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.665774 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.665898 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.666031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.666156 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.666280 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.666403 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.666574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.666701 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.666841 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.666967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.667105 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.667232 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.667359 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.667486 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.667623 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.667750 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.667876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.668000 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.668125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.668252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.668515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.668640 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.668775 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.668904 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.669031 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.669031 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.669279 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.669405 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.669531 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.669657 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.669784 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.669920 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.670046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.670174 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.670301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.670427 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.670581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.670715 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.670845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.670974 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.671101 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.671230 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.671363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.671499 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.671628 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.671755 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.671883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.672011 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.672144 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.672274 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.672411 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.672411 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.672652 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.672779 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.672917 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.673046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.673188 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.673326 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.673450 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.673618 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.673754 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.673879 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.674004 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.674137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.674270 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.674401 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.674561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.674561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.674801 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.674929 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.675054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.675182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.675319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.675446 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.675578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.675704 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.675828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.675970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.676103 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.676230 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.676355 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.676481 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.676606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.676733 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.676866 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.676992 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.677117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.677244 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.677244 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.677484 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.677618 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.677618 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.677866 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.677992 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.678118 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.678245 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.678384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.678532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.678659 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.678787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.678915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.679043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.679180 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.679310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.679447 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.679602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.679729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.679857 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.680000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.680130 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.680257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.680386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.680514 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.680643 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.680779 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.680908 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.681035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.681160 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.681299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.681425 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.681557 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.681682 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.681806 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.681929 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.682053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.682176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.682307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.682432 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.682432 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.682645 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.682645 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.682884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.682884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.683150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.683232 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.683395 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.683530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.683654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.683750 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.683867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.683990 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.684114 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.684237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.684360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.684491 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.684626 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.684751 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.684875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.684999 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.685123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.685255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.685380 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.685505 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.685632 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.685755 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.685880 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.686012 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.686135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.686259 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.686392 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.686555 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.686682 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.686821 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.686904 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.687023 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.687145 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.687228 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] role.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.687290 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.687356 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] role.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.687417 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.687474 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.687532 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.687589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.687646 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.687712 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.687773 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.687906 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.687974 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.687974 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.688319 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.688447 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.688572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.688698 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.688855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.688998 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.689077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.689140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.689199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.689360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] role.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.689446 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.689552 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] role.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.689688 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.689814 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.689897 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.690012 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.690136 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.690218 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.690369 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.690492 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.690492 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.690731 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.690799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.690859 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.690859 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.690859 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.691109 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.691191 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.691358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.691483 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.691619 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.691793 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.691920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.692045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.692179 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.692328 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.692328 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.692580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.692705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.692705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.692945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.693215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.693215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.693564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.693692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.693692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.693927 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.694070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.694070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.694305 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.694432 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.694629 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.694755 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.694891 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.694891 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.695131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.695131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.695371 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.695499 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.695635 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.695772 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.695898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.696031 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.696160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.696288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.696423 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.696550 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.696675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.696800 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.696926 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.697053 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.697187 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.697313 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.697440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.697440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.697688 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.697814 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.697947 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.698073 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.698200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.698200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.698200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.698570 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.698707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.698832 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.698832 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.699064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.699189 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.699322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.connection = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.699458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.699595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.699719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.699842 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.699965 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.700089 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.700212 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.700344 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.700344 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.700578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.700702 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.700825 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.700957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.701089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.701213 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.701337 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.701461 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.701586 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.701710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.701841 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.701965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.702088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.702212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.702336 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.702460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.702644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.702788 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.702914 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.703039 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.703166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.703291 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.703422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.703551 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.703679 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.connection = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.703814 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.703814 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81746) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 23 05:58:36.704061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.704061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.704061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.704061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.704061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.704061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.704061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.704061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.704061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.705847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.705847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.705847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.705847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.705847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.705847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.705847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.705847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.705847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 05:58:36.706908 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81747) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 23 05:58:36.735024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.735416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.735639 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.735909 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.736131 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.736353 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.744885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.745585 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.774722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.775122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.776514 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.776742 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.830803 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.831252 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.879909 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.880158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.882369 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:36.882631 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.039652 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.039880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.040099 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.040347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.040527 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.040737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.041776 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.042013 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.042240 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.042656 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.042750 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.042916 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.066207 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.066426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.066538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.066675 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.066762 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.066896 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.067001 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.067096 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.067173 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.067315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.067408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.067527 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.067611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.067738 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.079262 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.081446 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.092862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.093181 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.093322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.093534 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.093749 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.093963 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.094046 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.094175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81747) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.094257 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.094447 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.094696 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.094913 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.095633 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.095992 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81746) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 05:58:37.112907 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.113115 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 05:58:37.113466 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.113669 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 05:58:37.113853 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.113965 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.114160 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 05:58:37.114275 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.114619 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.114743 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.114826 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 05:58:37.115012 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.115091 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.115320 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.115842 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.116105 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.124694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.124976 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.125253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.125543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.126123 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.126443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.126755 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.127215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.133128 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.134087 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.136066 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.136356 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.136647 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.136939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.137022 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.137228 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.137309 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.137496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.137577 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.137774 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.137901 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.138055 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.138137 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.138321 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.138443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.138618 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.138752 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.138893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.139031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.139275 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.139552 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.139823 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.162828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.162928 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.164288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.164386 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.164571 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.164653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.169218 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.169363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.169503 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.169654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.169787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.169943 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.176762 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.177047 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.177289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.177581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.181125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.181414 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.181696 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.181790 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.181978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.182064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.182346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.182443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.182680 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.182760 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.182834 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.182907 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.183065 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.183156 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.183367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.183645 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.200566 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.200842 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.201368 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.201656 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.204735 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.206095 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.207954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.208252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.209339 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.209637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.212422 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.213857 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.215575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.217025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.218868 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.219170 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.219467 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.219760 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.220046 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.220859 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.220952 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.221382 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.221738 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.222043 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.222337 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.222665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.237409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.237701 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.237970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.238237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.239510 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.239801 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.240073 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.240340 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.245329 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.246865 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.247145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.247517 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.249056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.249338 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.249760 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.250045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.250337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.250652 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.250945 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.252383 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.252676 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.252976 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.253279 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.253575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.262580 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.262866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.263149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.263438 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.263720 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.263999 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.264276 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.264947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.265234 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.265523 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.265810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.266098 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.266400 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.266722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.279603 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.282380 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.283142 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.286209 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.286370 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.287651 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.289116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.289317 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.289400 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.289682 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.290996 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.292263 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.292558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.292854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.296417 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.297870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.300401 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.301697 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.301877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.301987 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.302274 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.302582 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.304910 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.305204 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.305497 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.305791 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.310455 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.310771 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.311062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.313718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.314093 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.314617 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.317908 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81746) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.318188 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.318470 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.318773 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.319060 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.319340 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.319613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.319879 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.320158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.320434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.320705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.320981 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.321254 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.321500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81747) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 05:58:37.321582 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81746) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.321791 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.322083 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.322366 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.322672 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.322958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.323235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.323517 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.323804 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.324082 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.324364 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.324640 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.324928 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.325192 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81747) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 05:58:37.356019 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x559076dc39c0 pid: 81746 (default app) Aug 23 05:58:37.359439 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfbcb6-4adb-4398-a959-23ebeb886105 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:37.359719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfbcb6-4adb-4398-a959-23ebeb886105 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:37.359984 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfbcb6-4adb-4398-a959-23ebeb886105 None None] PATH_INFO: `/v3` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:37.360408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x559076dc39c0 pid: 81747 (default app) Aug 23 05:58:37.364761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Tue Aug 23 05:58:37 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:58:44.021070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a0182-7d1b-491e-ab48-56dc838c67fd None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:44.021623 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a0182-7d1b-491e-ab48-56dc838c67fd None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:44.021623 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a0182-7d1b-491e-ab48-56dc838c67fd None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:44.025910 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:58:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:58:44.033293 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd014d5-48d8-4cab-a582-ee0a77ff6c42 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:44.033728 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd014d5-48d8-4cab-a582-ee0a77ff6c42 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:44.033819 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd014d5-48d8-4cab-a582-ee0a77ff6c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:44.103992 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: INFO dbcounter [None req-1dd014d5-48d8-4cab-a582-ee0a77ff6c42 None None] Registered counter for database keystone Aug 23 05:58:44.123853 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG oslo_db.sqlalchemy.engines [None req-1dd014d5-48d8-4cab-a582-ee0a77ff6c42 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81746) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 23 05:58:44.130052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writer thread running {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 23 05:58:44.316357 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG passlib.handlers.bcrypt [None req-1dd014d5-48d8-4cab-a582-ee0a77ff6c42 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81746) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 23 05:58:44.316783 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG passlib.handlers.bcrypt [None req-1dd014d5-48d8-4cab-a582-ee0a77ff6c42 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81746) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 23 05:58:44.371741 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1dd014d5-48d8-4cab-a582-ee0a77ff6c42 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:58:44.373343 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1dd014d5-48d8-4cab-a582-ee0a77ff6c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:58:44.503576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:58:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 23 05:58:44.977741 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a0182-7d1b-491e-ab48-56dc838c67fd None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:45.047442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: INFO dbcounter [None req-ae0a0182-7d1b-491e-ab48-56dc838c67fd None None] Registered counter for database keystone Aug 23 05:58:45.070359 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG oslo_db.sqlalchemy.engines [None req-ae0a0182-7d1b-491e-ab48-56dc838c67fd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81747) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 23 05:58:45.076554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writer thread running {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 23 05:58:45.200023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a0182-7d1b-491e-ab48-56dc838c67fd None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:45.204418 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c80eac-9c86-48db-a558-270721711da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e3a96fd05742d9a58f3a78e19ef4f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:45.204993 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c80eac-9c86-48db-a558-270721711da9 admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:45.205265 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c80eac-9c86-48db-a558-270721711da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:45.205533 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c80eac-9c86-48db-a558-270721711da9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:45.213930 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c80eac-9c86-48db-a558-270721711da9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:45.266221 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: WARNING keystone.common.rbac_enforcer.enforcer [None req-52c80eac-9c86-48db-a558-270721711da9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 23 05:58:45.266622 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c80eac-9c86-48db-a558-270721711da9 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:45.276907 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Tue Aug 23 05:58:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 05:58:45.283367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd014d5-48d8-4cab-a582-ee0a77ff6c42 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:45.297525 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd014d5-48d8-4cab-a582-ee0a77ff6c42 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:45.302250 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14817c-a652-405f-902e-ecb27659c5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e3a96fd05742d9a58f3a78e19ef4f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:45.303368 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14817c-a652-405f-902e-ecb27659c5fa admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:45.303648 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14817c-a652-405f-902e-ecb27659c5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:45.303913 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14817c-a652-405f-902e-ecb27659c5fa admin admin] PATH_INFO: `/v3/projects/a1e3a96fd05742d9a58f3a78e19ef4f6` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:45.305465 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14817c-a652-405f-902e-ecb27659c5fa admin admin] RBAC: Authorizing `identity:get_project(project_id=a1e3a96fd05742d9a58f3a78e19ef4f6)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:45.358674 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4a14817c-a652-405f-902e-ecb27659c5fa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 23 05:58:45.359012 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14817c-a652-405f-902e-ecb27659c5fa admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:45.364363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Tue Aug 23 05:58:45 2022] GET /identity/v3/projects/a1e3a96fd05742d9a58f3a78e19ef4f6 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:58:47.314904 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270add1-04e1-449f-9375-fc465dc61dd4 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:47.315462 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270add1-04e1-449f-9375-fc465dc61dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:47.315462 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270add1-04e1-449f-9375-fc465dc61dd4 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:47.319580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:58:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:58:47.325151 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094e9a7-cc1c-4928-b68a-fce420b4cede None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:47.325562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094e9a7-cc1c-4928-b68a-fce420b4cede None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:47.325693 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094e9a7-cc1c-4928-b68a-fce420b4cede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:47.346259 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1094e9a7-cc1c-4928-b68a-fce420b4cede None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:58:47.347800 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1094e9a7-cc1c-4928-b68a-fce420b4cede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:58:47.355605 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:58:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 23 05:58:47.625420 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270add1-04e1-449f-9375-fc465dc61dd4 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:47.637947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270add1-04e1-449f-9375-fc465dc61dd4 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:47.642879 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3511fb-d6e4-4c2d-a978-b89846741d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e3a96fd05742d9a58f3a78e19ef4f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:47.643477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3511fb-d6e4-4c2d-a978-b89846741d3f admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:47.643752 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3511fb-d6e4-4c2d-a978-b89846741d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:47.644023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3511fb-d6e4-4c2d-a978-b89846741d3f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:47.652048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3511fb-d6e4-4c2d-a978-b89846741d3f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:47.653832 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3511fb-d6e4-4c2d-a978-b89846741d3f admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:47.662904 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Tue Aug 23 05:58:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 05:58:47.669000 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094e9a7-cc1c-4928-b68a-fce420b4cede None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:47.678006 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094e9a7-cc1c-4928-b68a-fce420b4cede None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:47.682314 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522cd458-03d3-472d-9c0a-704097b9fa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e3a96fd05742d9a58f3a78e19ef4f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:47.682966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cd458-03d3-472d-9c0a-704097b9fa50 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:47.683245 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cd458-03d3-472d-9c0a-704097b9fa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:47.683514 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cd458-03d3-472d-9c0a-704097b9fa50 admin admin] PATH_INFO: `/v3/users/c922803daa44438cbb8185e3898e3dc9` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:47.684830 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522cd458-03d3-472d-9c0a-704097b9fa50 admin admin] RBAC: Authorizing `identity:get_user(user_id=c922803daa44438cbb8185e3898e3dc9)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:47.686679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522cd458-03d3-472d-9c0a-704097b9fa50 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:47.688742 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Tue Aug 23 05:58:47 2022] GET /identity/v3/users/c922803daa44438cbb8185e3898e3dc9 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:58:49.699725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c8cde-6fc4-4ee2-98a4-b70388fd4cf5 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:49.700254 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c8cde-6fc4-4ee2-98a4-b70388fd4cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:49.700254 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c8cde-6fc4-4ee2-98a4-b70388fd4cf5 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:49.701600 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:58:49.707726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28e8fa-6b3d-43a6-a3eb-0ecd539307bc None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:49.708139 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28e8fa-6b3d-43a6-a3eb-0ecd539307bc None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:49.708283 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28e8fa-6b3d-43a6-a3eb-0ecd539307bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:49.720579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541260a-a23e-4c7c-9c0f-9344fe15bbaa None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:49.720871 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541260a-a23e-4c7c-9c0f-9344fe15bbaa None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:49.721145 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541260a-a23e-4c7c-9c0f-9344fe15bbaa None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:49.722493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:58:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:58:49.726970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-4e28e8fa-6b3d-43a6-a3eb-0ecd539307bc None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:58:49.727558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dab877-97fe-410d-bf7c-29e30a15cdb6 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:49.727841 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dab877-97fe-410d-bf7c-29e30a15cdb6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:49.728143 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dab877-97fe-410d-bf7c-29e30a15cdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:49.728253 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4e28e8fa-6b3d-43a6-a3eb-0ecd539307bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:58:49.771731 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG passlib.handlers.bcrypt [None req-c6dab877-97fe-410d-bf7c-29e30a15cdb6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81747) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 23 05:58:49.772146 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG passlib.handlers.bcrypt [None req-c6dab877-97fe-410d-bf7c-29e30a15cdb6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81747) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 23 05:58:49.808874 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:58:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:58:49.818881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-c6dab877-97fe-410d-bf7c-29e30a15cdb6 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:58:49.820411 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c6dab877-97fe-410d-bf7c-29e30a15cdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:58:49.919611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:58:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:58:50.075633 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28e8fa-6b3d-43a6-a3eb-0ecd539307bc None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:50.099646 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:50.180691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28e8fa-6b3d-43a6-a3eb-0ecd539307bc None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:50.197531 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dab877-97fe-410d-bf7c-29e30a15cdb6 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:50.215495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ed1e2-d407-4bdf-9c16-66888ccffc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:50.215998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ed1e2-d407-4bdf-9c16-66888ccffc10 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:50.216278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ed1e2-d407-4bdf-9c16-66888ccffc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:50.216556 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ed1e2-d407-4bdf-9c16-66888ccffc10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:50.227998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ed1e2-d407-4bdf-9c16-66888ccffc10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-e71ed1e2-d407-4bdf-9c16-66888ccffc10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:58:50.237391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:58:50.238299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:58:50.239409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:58:50.239409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:58:50.239409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:58:50.239409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:58:50.239409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:58:50.239409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:58:50.239409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:58:50.239409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:58:50.239409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:58:50.239409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:58:50.239409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:58:50.239409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:58:50.239409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 05:58:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:58:50.244717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ed1e2-d407-4bdf-9c16-66888ccffc10 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:50.304088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dab877-97fe-410d-bf7c-29e30a15cdb6 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:50.306440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:50.327296 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:50.346853 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ed1e2-d407-4bdf-9c16-66888ccffc10 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:50.347253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3862696-f705-471e-9e3f-ab5f4f6242cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:50.348112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3862696-f705-471e-9e3f-ab5f4f6242cd None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:50.348397 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3862696-f705-471e-9e3f-ab5f4f6242cd None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:50.348668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3862696-f705-471e-9e3f-ab5f4f6242cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:50.381251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24cdf4-2de6-49b1-86dd-945f8df5e8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:50.381858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24cdf4-2de6-49b1-86dd-945f8df5e8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:50.382134 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24cdf4-2de6-49b1-86dd-945f8df5e8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:50.382402 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24cdf4-2de6-49b1-86dd-945f8df5e8e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:50.383106 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24cdf4-2de6-49b1-86dd-945f8df5e8e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:58:50.383404 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24cdf4-2de6-49b1-86dd-945f8df5e8e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:50.384799 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24cdf4-2de6-49b1-86dd-945f8df5e8e1 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:50.396941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:58:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:58:50.402466 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24cdf4-2de6-49b1-86dd-945f8df5e8e1 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:50.416382 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3862696-f705-471e-9e3f-ab5f4f6242cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:50.417916 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3862696-f705-471e-9e3f-ab5f4f6242cd None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:50.516883 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24cdf4-2de6-49b1-86dd-945f8df5e8e1 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:50.531858 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:58:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 05:58:50.538179 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3862696-f705-471e-9e3f-ab5f4f6242cd None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:50.541774 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:50.552360 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbd0b2-db03-468c-bf2f-04d8641894a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:50.553009 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbd0b2-db03-468c-bf2f-04d8641894a0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:50.553285 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbd0b2-db03-468c-bf2f-04d8641894a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:50.553558 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbd0b2-db03-468c-bf2f-04d8641894a0 None admin] PATH_INFO: `/v3/users/c922803daa44438cbb8185e3898e3dc9` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:50.554744 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbd0b2-db03-468c-bf2f-04d8641894a0 None admin] RBAC: Authorizing `identity:get_user(user_id=c922803daa44438cbb8185e3898e3dc9)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:50.556215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbd0b2-db03-468c-bf2f-04d8641894a0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:50.557836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:58:50 2022] GET /identity/v3/users/c922803daa44438cbb8185e3898e3dc9 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:58:50.563998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbd0b2-db03-468c-bf2f-04d8641894a0 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:50.579766 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:50.642213 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3862696-f705-471e-9e3f-ab5f4f6242cd None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:50.671235 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbd0b2-db03-468c-bf2f-04d8641894a0 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:50.675835 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83892f4-fd93-4ce6-9426-1d7cb571008b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:50.676698 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83892f4-fd93-4ce6-9426-1d7cb571008b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:50.676979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83892f4-fd93-4ce6-9426-1d7cb571008b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:50.677247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83892f4-fd93-4ce6-9426-1d7cb571008b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:50.696139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83892f4-fd93-4ce6-9426-1d7cb571008b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:50.697613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83892f4-fd93-4ce6-9426-1d7cb571008b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:50.705389 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9a057-7398-4109-b317-a407e711fffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:50.706014 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9a057-7398-4109-b317-a407e711fffe None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:50.706288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9a057-7398-4109-b317-a407e711fffe None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:50.706589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9a057-7398-4109-b317-a407e711fffe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:50.707714 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9a057-7398-4109-b317-a407e711fffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:50.709157 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9a057-7398-4109-b317-a407e711fffe None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:50.710725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 10/18] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:58:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:58:50.716426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9a057-7398-4109-b317-a407e711fffe None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-d83892f4-fd93-4ce6-9426-1d7cb571008b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 05:58:50.716751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:58:50.718065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 05:58:50.719104 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 05:58:50.719104 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 05:58:50.719104 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:58:50.719104 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 9/19] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:58:50.723772 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83892f4-fd93-4ce6-9426-1d7cb571008b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:50.782318 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:50.806780 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:50.831024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83892f4-fd93-4ce6-9426-1d7cb571008b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:50.833654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9a057-7398-4109-b317-a407e711fffe None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:50.867155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d7b20-1bc4-45da-b4b3-61d539f46dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:50.867773 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d7b20-1bc4-45da-b4b3-61d539f46dec None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:50.868309 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d7b20-1bc4-45da-b4b3-61d539f46dec None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:50.868581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d7b20-1bc4-45da-b4b3-61d539f46dec None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:50.869221 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d7b20-1bc4-45da-b4b3-61d539f46dec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:58:50.869524 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d7b20-1bc4-45da-b4b3-61d539f46dec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:50.870955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d7b20-1bc4-45da-b4b3-61d539f46dec None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:50.871431 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee222c95-4098-47e1-9174-d152a3cf1dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:50.872033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee222c95-4098-47e1-9174-d152a3cf1dae None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:50.872317 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee222c95-4098-47e1-9174-d152a3cf1dae None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:50.872593 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee222c95-4098-47e1-9174-d152a3cf1dae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:50.873315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee222c95-4098-47e1-9174-d152a3cf1dae None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.domain.id=default, user.id=c922803daa44438cbb8185e3898e3dc9) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:58:50.873746 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee222c95-4098-47e1-9174-d152a3cf1dae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:50.875205 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee222c95-4098-47e1-9174-d152a3cf1dae None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:50.890192 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 11/20] 10.222.0.20 () {56 vars in 1322 bytes} [Tue Aug 23 05:58:50 2022] GET /identity/v3/role_assignments?user.id=c922803daa44438cbb8185e3898e3dc9&scope.domain.id=default&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 233 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:58:50.890392 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 10/21] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 05:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:58:53.118381 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1151f6-1b16-444a-835b-c821f72ea910 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:53.119412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1151f6-1b16-444a-835b-c821f72ea910 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:53.119896 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1151f6-1b16-444a-835b-c821f72ea910 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:53.121430 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 12/22] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:58:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:58:53.127579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9eb56-ad70-41f1-b1f6-c778bce4021d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:53.128003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9eb56-ad70-41f1-b1f6-c778bce4021d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:53.128178 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9eb56-ad70-41f1-b1f6-c778bce4021d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:53.144242 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006547b1-7317-4bd0-8791-17cce1d3e440 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:53.145002 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006547b1-7317-4bd0-8791-17cce1d3e440 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:53.145564 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006547b1-7317-4bd0-8791-17cce1d3e440 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:53.147794 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 13/23] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:58:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:58:53.153476 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815f39a-82db-4596-abd8-0535d3fdbe35 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:53.154029 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815f39a-82db-4596-abd8-0535d3fdbe35 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:53.154492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815f39a-82db-4596-abd8-0535d3fdbe35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:53.155985 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-a1d9eb56-ad70-41f1-b1f6-c778bce4021d None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:58:53.157418 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a1d9eb56-ad70-41f1-b1f6-c778bce4021d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:58:53.182042 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5815f39a-82db-4596-abd8-0535d3fdbe35 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:58:53.183643 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5815f39a-82db-4596-abd8-0535d3fdbe35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:58:53.318310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 14/24] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:58:53.320812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 11/25] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:58:53.612613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815f39a-82db-4596-abd8-0535d3fdbe35 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:53.708346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:53.744531 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815f39a-82db-4596-abd8-0535d3fdbe35 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:53.784331 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c35d0-6151-4495-a50d-822b0963896e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:53.785310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c35d0-6151-4495-a50d-822b0963896e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:53.785779 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c35d0-6151-4495-a50d-822b0963896e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:53.786280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c35d0-6151-4495-a50d-822b0963896e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:53.786824 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07b8e9-6680-4b91-8488-ae8ed20301e9 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:53.787107 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07b8e9-6680-4b91-8488-ae8ed20301e9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:53.787378 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07b8e9-6680-4b91-8488-ae8ed20301e9 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:53.788714 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 12/26] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:58:53.793771 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bc806-0a6d-43d3-92a2-02bb0dfe77de None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:53.798689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bc806-0a6d-43d3-92a2-02bb0dfe77de None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:53.799212 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bc806-0a6d-43d3-92a2-02bb0dfe77de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:53.807199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c35d0-6151-4495-a50d-822b0963896e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:53.813562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c35d0-6151-4495-a50d-822b0963896e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:53.831384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-506bc806-0a6d-43d3-92a2-02bb0dfe77de None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:58:53.832947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-506bc806-0a6d-43d3-92a2-02bb0dfe77de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-2d4c35d0-6151-4495-a50d-822b0963896e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 05:58:53.836855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:58:53.837685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 05:58:53.838465 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 05:58:53.838465 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 05:58:53.838465 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:58:53.838465 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 15/27] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:58:53.850921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c35d0-6151-4495-a50d-822b0963896e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:53.885184 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:53.915190 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 13/28] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:58:53.916784 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bc806-0a6d-43d3-92a2-02bb0dfe77de None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:54.038433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bc806-0a6d-43d3-92a2-02bb0dfe77de None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:54.071544 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c35d0-6151-4495-a50d-822b0963896e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:54.104147 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146a15f-cdd0-4613-a983-d4456fbcf2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:54.105133 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146a15f-cdd0-4613-a983-d4456fbcf2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:54.105604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146a15f-cdd0-4613-a983-d4456fbcf2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:54.105827 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd312ee-867f-43db-bcec-a39316547297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:54.106320 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146a15f-cdd0-4613-a983-d4456fbcf2a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:54.106427 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd312ee-867f-43db-bcec-a39316547297 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:54.106749 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd312ee-867f-43db-bcec-a39316547297 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:54.107018 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd312ee-867f-43db-bcec-a39316547297 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:54.107674 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146a15f-cdd0-4613-a983-d4456fbcf2a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:58:54.108158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146a15f-cdd0-4613-a983-d4456fbcf2a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:54.110239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146a15f-cdd0-4613-a983-d4456fbcf2a6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:54.118611 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:54.119510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd312ee-867f-43db-bcec-a39316547297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-7cd312ee-867f-43db-bcec-a39316547297 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:58:54.124295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:58:54.125121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:58:54.126044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:58:54.126044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:58:54.126044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:58:54.126044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:58:54.126044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:58:54.126044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:58:54.126044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:58:54.126044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:58:54.126044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:58:54.126044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:58:54.126044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:58:54.126044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:58:54.126044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 16/29] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 05:58:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:58:54.127413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5829d49-2cde-4f20-9260-7f1d843a5a39 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:54.127692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5829d49-2cde-4f20-9260-7f1d843a5a39 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:54.127963 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5829d49-2cde-4f20-9260-7f1d843a5a39 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:54.129636 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 17/30] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:58:54.134567 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5829d49-2cde-4f20-9260-7f1d843a5a39 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:54.146114 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 14/31] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 05:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:58:54.147980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a291f-93e9-4ca7-99f8-76c5340fe93d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:54.148400 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a291f-93e9-4ca7-99f8-76c5340fe93d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:54.148825 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a291f-93e9-4ca7-99f8-76c5340fe93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:54.184101 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-7b7a291f-93e9-4ca7-99f8-76c5340fe93d None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:58:54.185473 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7b7a291f-93e9-4ca7-99f8-76c5340fe93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:58:54.240442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:54.272996 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5829d49-2cde-4f20-9260-7f1d843a5a39 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:54.293194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 15/32] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:58:54.294341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a291f-93e9-4ca7-99f8-76c5340fe93d None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:54.324924 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8e625-e862-49f6-bbf1-472ca58fdf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:54.325835 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8e625-e862-49f6-bbf1-472ca58fdf41 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:54.326119 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8e625-e862-49f6-bbf1-472ca58fdf41 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:54.326388 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8e625-e862-49f6-bbf1-472ca58fdf41 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:54.327087 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8e625-e862-49f6-bbf1-472ca58fdf41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:58:54.327389 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8e625-e862-49f6-bbf1-472ca58fdf41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:54.329688 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8e625-e862-49f6-bbf1-472ca58fdf41 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:54.339498 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 18/33] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:58:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:58:54.340560 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8e625-e862-49f6-bbf1-472ca58fdf41 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:54.403068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:54.457403 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8e625-e862-49f6-bbf1-472ca58fdf41 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:54.495450 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec47c2-435f-4e6d-941e-067f3808af50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:54.496366 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec47c2-435f-4e6d-941e-067f3808af50 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:54.496672 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec47c2-435f-4e6d-941e-067f3808af50 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:54.496945 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec47c2-435f-4e6d-941e-067f3808af50 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:54.497615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec47c2-435f-4e6d-941e-067f3808af50 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:54.499068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec47c2-435f-4e6d-941e-067f3808af50 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:54.514573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a291f-93e9-4ca7-99f8-76c5340fe93d None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:54.587169 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eda98-d12b-44f9-95c7-c01f5b480611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:54.588206 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eda98-d12b-44f9-95c7-c01f5b480611 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:54.588696 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eda98-d12b-44f9-95c7-c01f5b480611 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:54.589205 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eda98-d12b-44f9-95c7-c01f5b480611 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:54.590497 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417eda98-d12b-44f9-95c7-c01f5b480611 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:54.591991 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417eda98-d12b-44f9-95c7-c01f5b480611 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:54.643080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 19/34] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Aug 23 05:58:54 2022] POST /identity/v3/roles => generated 219 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 05:58:54.644095 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec47c2-435f-4e6d-941e-067f3808af50 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:54.761877 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec47c2-435f-4e6d-941e-067f3808af50 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:54.766037 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 16/35] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 23 05:58:54 2022] POST /identity/v3/projects => generated 301 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 05:58:54.768089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eda98-d12b-44f9-95c7-c01f5b480611 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:54.772573 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:54.801833 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466a009-c605-4e55-99a5-a3efb099a018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:54.802453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466a009-c605-4e55-99a5-a3efb099a018 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:54.803021 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466a009-c605-4e55-99a5-a3efb099a018 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:54.803395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466a009-c605-4e55-99a5-a3efb099a018 None admin] PATH_INFO: `/v3/users/c922803daa44438cbb8185e3898e3dc9` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:54.804321 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:54.804975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466a009-c605-4e55-99a5-a3efb099a018 None admin] RBAC: Authorizing `identity:get_user(user_id=c922803daa44438cbb8185e3898e3dc9)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:54.806719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466a009-c605-4e55-99a5-a3efb099a018 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:54.808562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 20/36] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:58:54 2022] GET /identity/v3/users/c922803daa44438cbb8185e3898e3dc9 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:58:54.814824 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466a009-c605-4e55-99a5-a3efb099a018 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:54.954917 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466a009-c605-4e55-99a5-a3efb099a018 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:54.970555 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eda98-d12b-44f9-95c7-c01f5b480611 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:55.004278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc468e93-f925-4c40-80f6-5aa7fbe9a3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:55.004937 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc468e93-f925-4c40-80f6-5aa7fbe9a3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:55.005224 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc468e93-f925-4c40-80f6-5aa7fbe9a3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:55.005500 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc468e93-f925-4c40-80f6-5aa7fbe9a3c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:55.007681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc468e93-f925-4c40-80f6-5aa7fbe9a3c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:55.008385 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc468e93-f925-4c40-80f6-5aa7fbe9a3c0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:55.013581 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 21/37] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:58:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:58:55.013920 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f4a22-55a3-49bc-b432-8f1c434f6274 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:55.014213 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f4a22-55a3-49bc-b432-8f1c434f6274 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:55.015436 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f4a22-55a3-49bc-b432-8f1c434f6274 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:55.017791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 22/38] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:58:55.019004 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f4a22-55a3-49bc-b432-8f1c434f6274 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:55.032814 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3124410-d540-4903-a139-049a0111b89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:55.034549 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3124410-d540-4903-a139-049a0111b89c None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:55.034832 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3124410-d540-4903-a139-049a0111b89c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:55.035107 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3124410-d540-4903-a139-049a0111b89c None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:55.036311 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3124410-d540-4903-a139-049a0111b89c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:55.038303 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3124410-d540-4903-a139-049a0111b89c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:55.068335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 17/39] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Aug 23 05:58:54 2022] POST /identity/v3/roles => generated 225 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 05:58:55.070131 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01dda7-e4f0-4082-bae9-832e8b1086df None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:55.070313 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01dda7-e4f0-4082-bae9-832e8b1086df None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:55.070611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01dda7-e4f0-4082-bae9-832e8b1086df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:55.100091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-af01dda7-e4f0-4082-bae9-832e8b1086df None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:58:55.101582 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:55.102433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-af01dda7-e4f0-4082-bae9-832e8b1086df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:58:55.195357 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f4a22-55a3-49bc-b432-8f1c434f6274 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:55.215434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 18/40] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:58:55.215434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862ceeb-ed4b-4337-825f-34e922bf11dc None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:55.215434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862ceeb-ed4b-4337-825f-34e922bf11dc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:55.215434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862ceeb-ed4b-4337-825f-34e922bf11dc None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:55.215434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 19/41] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:58:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:58:55.221523 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcb7b7-95ae-42a0-ac8e-1ccc5cdbd9a2 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:55.221806 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcb7b7-95ae-42a0-ac8e-1ccc5cdbd9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:55.222084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcb7b7-95ae-42a0-ac8e-1ccc5cdbd9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:55.252327 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-02dcb7b7-95ae-42a0-ac8e-1ccc5cdbd9a2 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:58:55.254537 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-02dcb7b7-95ae-42a0-ac8e-1ccc5cdbd9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:58:55.256784 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:55.256977 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2199590-7214-454e-a2e8-8ca1fcf94595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:55.258637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2199590-7214-454e-a2e8-8ca1fcf94595 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:55.258840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2199590-7214-454e-a2e8-8ca1fcf94595 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:55.259175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2199590-7214-454e-a2e8-8ca1fcf94595 None admin] PATH_INFO: `/v3/domains/default/users/c922803daa44438cbb8185e3898e3dc9/roles/f6acb6fdca5a44de9ba00b2b0d4d3202` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:55.261242 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2199590-7214-454e-a2e8-8ca1fcf94595 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c922803daa44438cbb8185e3898e3dc9, role_id=f6acb6fdca5a44de9ba00b2b0d4d3202)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:55.262969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2199590-7214-454e-a2e8-8ca1fcf94595 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:55.292874 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 23/42] 10.222.0.20 () {58 vars in 1361 bytes} [Tue Aug 23 05:58:55 2022] PUT /identity/v3/domains/default/users/c922803daa44438cbb8185e3898e3dc9/roles/f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 05:58:55.366422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 20/43] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:58:55.546483 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2199590-7214-454e-a2e8-8ca1fcf94595 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:55.704774 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcb7b7-95ae-42a0-ac8e-1ccc5cdbd9a2 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:55.791801 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2199590-7214-454e-a2e8-8ca1fcf94595 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:55.794312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:55.832396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcb7b7-95ae-42a0-ac8e-1ccc5cdbd9a2 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:55.839641 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6050031-3994-4de1-9eb5-9306f43b3f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:55.840271 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6050031-3994-4de1-9eb5-9306f43b3f89 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:55.840550 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6050031-3994-4de1-9eb5-9306f43b3f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:55.840823 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6050031-3994-4de1-9eb5-9306f43b3f89 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:55.841515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6050031-3994-4de1-9eb5-9306f43b3f89 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:55.843048 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6050031-3994-4de1-9eb5-9306f43b3f89 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:55.861948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 24/44] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Aug 23 05:58:55 2022] POST /identity/v3/roles => generated 223 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 05:58:55.878251 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d19feb-6ba8-4332-af18-80104fd533bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:55.879328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d19feb-6ba8-4332-af18-80104fd533bc None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:55.879786 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d19feb-6ba8-4332-af18-80104fd533bc None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:55.880277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d19feb-6ba8-4332-af18-80104fd533bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:55.881799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d19feb-6ba8-4332-af18-80104fd533bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:55.883597 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d19feb-6ba8-4332-af18-80104fd533bc None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:55.885637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 21/45] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:58:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:58:55.893510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6050031-3994-4de1-9eb5-9306f43b3f89 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:56.070661 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6050031-3994-4de1-9eb5-9306f43b3f89 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:56.110252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f1df9-7a0a-4d22-918a-bdab3d1a6ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:56.110821 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f1df9-7a0a-4d22-918a-bdab3d1a6ba6 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:56.111103 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f1df9-7a0a-4d22-918a-bdab3d1a6ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:56.111378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f1df9-7a0a-4d22-918a-bdab3d1a6ba6 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:56.112027 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f1df9-7a0a-4d22-918a-bdab3d1a6ba6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:56.113564 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f1df9-7a0a-4d22-918a-bdab3d1a6ba6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:56.146580 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 25/46] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 23 05:58:55 2022] POST /identity/v3/projects => generated 312 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 05:58:57.280309 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2c420-8066-4a8c-8d44-32cf24659a02 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:57.280829 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2c420-8066-4a8c-8d44-32cf24659a02 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:57.280829 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2c420-8066-4a8c-8d44-32cf24659a02 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:57.282841 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 22/47] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:58:57.288371 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16519d9-c641-41fb-8f54-1a459b44edce None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:57.288830 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16519d9-c641-41fb-8f54-1a459b44edce None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:57.288931 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16519d9-c641-41fb-8f54-1a459b44edce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:57.308224 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a16519d9-c641-41fb-8f54-1a459b44edce None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:58:57.309750 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a16519d9-c641-41fb-8f54-1a459b44edce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:58:57.327871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:57.394192 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 26/48] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:58:57.661553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2c420-8066-4a8c-8d44-32cf24659a02 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:57.670591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:57.768190 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2c420-8066-4a8c-8d44-32cf24659a02 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:57.806164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e8910-7afe-4f15-8f67-2b861758f4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:57.806832 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e8910-7afe-4f15-8f67-2b861758f4da None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:57.807119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e8910-7afe-4f15-8f67-2b861758f4da None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:57.807419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e8910-7afe-4f15-8f67-2b861758f4da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:57.819235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e8910-7afe-4f15-8f67-2b861758f4da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-989e8910-7afe-4f15-8f67-2b861758f4da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:58:57.825467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:58:57.826449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:58:57.827322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:58:57.827322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:58:57.827322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:58:57.827322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:58:57.827322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:58:57.827322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:58:57.827322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:58:57.827322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:58:57.827322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:58:57.827322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:58:57.827322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:58:57.827322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:58:57.827322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 23/49] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 05:58:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:58:57.835428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16519d9-c641-41fb-8f54-1a459b44edce None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:57.937363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16519d9-c641-41fb-8f54-1a459b44edce None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:57.976528 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50affebb-c25a-4352-85e8-201907043233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:57.977138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50affebb-c25a-4352-85e8-201907043233 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:57.977419 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50affebb-c25a-4352-85e8-201907043233 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:57.977691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50affebb-c25a-4352-85e8-201907043233 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:57.978356 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50affebb-c25a-4352-85e8-201907043233 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:58:57.978693 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50affebb-c25a-4352-85e8-201907043233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:57.980091 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50affebb-c25a-4352-85e8-201907043233 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:57.988991 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 27/50] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:58:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:58:57.994786 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e8910-7afe-4f15-8f67-2b861758f4da None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:58.064815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:58.078097 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedf889-f55b-4381-85f2-d995fda2be01 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:58.078483 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedf889-f55b-4381-85f2-d995fda2be01 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:58.078793 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acedf889-f55b-4381-85f2-d995fda2be01 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:58.080137 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 28/51] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:58:58.085270 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e851d-f34d-4a3a-8818-b8b6b32ebe3f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:58.085555 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e851d-f34d-4a3a-8818-b8b6b32ebe3f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:58.085829 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e851d-f34d-4a3a-8818-b8b6b32ebe3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:58.102566 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e8910-7afe-4f15-8f67-2b861758f4da None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:58.104595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1e3e851d-f34d-4a3a-8818-b8b6b32ebe3f None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:58:58.105794 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1e3e851d-f34d-4a3a-8818-b8b6b32ebe3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:58:58.107889 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:58.138713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13080175-1c8d-47c4-a7a6-9f2f9ef1c340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:58.139350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13080175-1c8d-47c4-a7a6-9f2f9ef1c340 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:58.139643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13080175-1c8d-47c4-a7a6-9f2f9ef1c340 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:58.139903 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13080175-1c8d-47c4-a7a6-9f2f9ef1c340 None admin] PATH_INFO: `/v3/users/c922803daa44438cbb8185e3898e3dc9` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:58.141297 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13080175-1c8d-47c4-a7a6-9f2f9ef1c340 None admin] RBAC: Authorizing `identity:get_user(user_id=c922803daa44438cbb8185e3898e3dc9)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:58.142798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13080175-1c8d-47c4-a7a6-9f2f9ef1c340 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:58.144432 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 24/52] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:58:57 2022] GET /identity/v3/users/c922803daa44438cbb8185e3898e3dc9 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:58:58.150110 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13080175-1c8d-47c4-a7a6-9f2f9ef1c340 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:58.179481 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 29/53] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:58:58.258565 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13080175-1c8d-47c4-a7a6-9f2f9ef1c340 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:58.293724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591f862-7e8a-4ba2-975f-604e32d48746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:58.294385 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591f862-7e8a-4ba2-975f-604e32d48746 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:58.294649 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591f862-7e8a-4ba2-975f-604e32d48746 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:58.294925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591f862-7e8a-4ba2-975f-604e32d48746 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:58.296054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591f862-7e8a-4ba2-975f-604e32d48746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:58.298307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591f862-7e8a-4ba2-975f-604e32d48746 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:58.299916 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 25/54] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:58:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:58:58.305906 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e851d-f34d-4a3a-8818-b8b6b32ebe3f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:58.411936 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e851d-f34d-4a3a-8818-b8b6b32ebe3f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:58.449113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd5d30-b3f4-498e-a8e4-ef477998c561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:58.449747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd5d30-b3f4-498e-a8e4-ef477998c561 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:58.450029 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd5d30-b3f4-498e-a8e4-ef477998c561 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:58.450304 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd5d30-b3f4-498e-a8e4-ef477998c561 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:58.451174 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd5d30-b3f4-498e-a8e4-ef477998c561 None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.domain.id=default, user.id=c922803daa44438cbb8185e3898e3dc9) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:58:58.451481 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd5d30-b3f4-498e-a8e4-ef477998c561 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:58.451776 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591f862-7e8a-4ba2-975f-604e32d48746 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:58.452951 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd5d30-b3f4-498e-a8e4-ef477998c561 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:58.455429 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:58.460144 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 30/55] 10.222.0.20 () {56 vars in 1322 bytes} [Tue Aug 23 05:58:58 2022] GET /identity/v3/role_assignments?user.id=c922803daa44438cbb8185e3898e3dc9&scope.domain.id=default&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:58:58.580764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591f862-7e8a-4ba2-975f-604e32d48746 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:58.619069 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5953c-e77f-4aaa-9d44-5ec171e6a7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:58.619712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5953c-e77f-4aaa-9d44-5ec171e6a7ec None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:58.619990 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5953c-e77f-4aaa-9d44-5ec171e6a7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:58.620264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5953c-e77f-4aaa-9d44-5ec171e6a7ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:58.621387 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5953c-e77f-4aaa-9d44-5ec171e6a7ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:58.622881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5953c-e77f-4aaa-9d44-5ec171e6a7ec None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:58.625250 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 26/56] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:58:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:58:58.630637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd5d30-b3f4-498e-a8e4-ef477998c561 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:58:58.633575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:58:58.742434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd5d30-b3f4-498e-a8e4-ef477998c561 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:58:58.783058 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5c153-b0eb-4f15-85af-98e021eb7b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:58:58.783669 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5c153-b0eb-4f15-85af-98e021eb7b56 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:58:58.783951 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5c153-b0eb-4f15-85af-98e021eb7b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:58:58.784225 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5c153-b0eb-4f15-85af-98e021eb7b56 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:58:58.784880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5c153-b0eb-4f15-85af-98e021eb7b56 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:58:58.786328 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5c153-b0eb-4f15-85af-98e021eb7b56 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:58:58.812246 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 31/57] 10.222.0.20 () {60 vars in 1140 bytes} [Tue Aug 23 05:58:58 2022] POST /identity/v3/projects => generated 298 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 05:59:00.703078 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798f2c4-0c28-447b-a81a-d1f60b736373 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:00.704291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798f2c4-0c28-447b-a81a-d1f60b736373 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:00.704429 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798f2c4-0c28-447b-a81a-d1f60b736373 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:00.706798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 27/58] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:00.713335 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014d2a0-591d-4c53-82a0-820078502041 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:00.713759 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014d2a0-591d-4c53-82a0-820078502041 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:00.713887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014d2a0-591d-4c53-82a0-820078502041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:00.732604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e014d2a0-591d-4c53-82a0-820078502041 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:00.733777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e014d2a0-591d-4c53-82a0-820078502041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:00.789610 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:00.809367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 32/59] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:01.081028 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798f2c4-0c28-447b-a81a-d1f60b736373 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:01.153196 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:01.185655 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798f2c4-0c28-447b-a81a-d1f60b736373 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:01.221910 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a7493-dcd7-4b18-943f-933e43de625f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:01.222464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a7493-dcd7-4b18-943f-933e43de625f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:01.222787 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a7493-dcd7-4b18-943f-933e43de625f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:01.223113 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a7493-dcd7-4b18-943f-933e43de625f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:01.224165 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a7493-dcd7-4b18-943f-933e43de625f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:01.225608 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a7493-dcd7-4b18-943f-933e43de625f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:01.227214 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 28/60] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:59:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:01.233147 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014d2a0-591d-4c53-82a0-820078502041 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:01.331634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014d2a0-591d-4c53-82a0-820078502041 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:01.365949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8523e5-7d0d-4252-b94f-fd381b5ef2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:01.366579 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8523e5-7d0d-4252-b94f-fd381b5ef2e9 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:01.366865 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8523e5-7d0d-4252-b94f-fd381b5ef2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:01.367142 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8523e5-7d0d-4252-b94f-fd381b5ef2e9 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:01.367789 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8523e5-7d0d-4252-b94f-fd381b5ef2e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:01.369506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8523e5-7d0d-4252-b94f-fd381b5ef2e9 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:01.398921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 33/61] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 05:59:01 2022] POST /identity/v3/users => generated 275 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 05:59:04.459293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682034a7-9770-4097-9db7-ac11cbcc8fe4 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:04.459809 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682034a7-9770-4097-9db7-ac11cbcc8fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:04.460057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682034a7-9770-4097-9db7-ac11cbcc8fe4 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:04.461635 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 29/62] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:04.467541 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4e956-dd4b-40e9-b807-fe4773c975eb None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:04.468006 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4e956-dd4b-40e9-b807-fe4773c975eb None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:04.468164 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4e956-dd4b-40e9-b807-fe4773c975eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:04.490928 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1bc4e956-dd4b-40e9-b807-fe4773c975eb None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:04.492138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1bc4e956-dd4b-40e9-b807-fe4773c975eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:04.539318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:04.591450 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 34/63] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:04.753719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07c84e-1db0-45ce-a252-4a925f1de5ce None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:04.754142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07c84e-1db0-45ce-a252-4a925f1de5ce None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:04.754312 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07c84e-1db0-45ce-a252-4a925f1de5ce None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:04.755802 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 30/64] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:59:04.761048 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4bcca-51dc-44b2-b200-775c7cce4d15 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:04.761334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4bcca-51dc-44b2-b200-775c7cce4d15 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:04.761618 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4bcca-51dc-44b2-b200-775c7cce4d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:04.783481 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a9f4bcca-51dc-44b2-b200-775c7cce4d15 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:04.784706 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a9f4bcca-51dc-44b2-b200-775c7cce4d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:04.884287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 35/65] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:59:04.915496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c3008-ecb5-4edc-9561-4ce4027e51ef None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:04.915808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c3008-ecb5-4edc-9561-4ce4027e51ef None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:04.916063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c3008-ecb5-4edc-9561-4ce4027e51ef None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:04.917428 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 31/66] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:59:04.923371 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712a1ac-9e65-4992-858d-d948f12ed341 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:04.923657 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712a1ac-9e65-4992-858d-d948f12ed341 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:04.923935 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712a1ac-9e65-4992-858d-d948f12ed341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:04.952613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a712a1ac-9e65-4992-858d-d948f12ed341 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:04.958535 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a712a1ac-9e65-4992-858d-d948f12ed341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:04.972346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c3008-ecb5-4edc-9561-4ce4027e51ef None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:05.059337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 36/67] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:05.060636 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722b8ee-59c9-4711-b370-6aa2a4045175 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:05.061148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722b8ee-59c9-4711-b370-6aa2a4045175 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:05.061620 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722b8ee-59c9-4711-b370-6aa2a4045175 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:05.063281 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 37/68] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:05.068843 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd1e7f-af17-4a80-a22b-dd387b2f47af None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:05.069364 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd1e7f-af17-4a80-a22b-dd387b2f47af None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:05.069839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd1e7f-af17-4a80-a22b-dd387b2f47af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:05.105188 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-07bd1e7f-af17-4a80-a22b-dd387b2f47af None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:05.106687 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-07bd1e7f-af17-4a80-a22b-dd387b2f47af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:05.148639 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:05.201032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 38/69] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:05.202602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc931e4-a0a4-4818-ac74-54b178ee543c None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:05.203167 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc931e4-a0a4-4818-ac74-54b178ee543c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:05.203608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc931e4-a0a4-4818-ac74-54b178ee543c None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:05.205294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 39/70] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:05.206245 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc931e4-a0a4-4818-ac74-54b178ee543c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:05.261330 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c3008-ecb5-4edc-9561-4ce4027e51ef None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:05.301351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24785edc-9c8f-4edf-a75f-a301657b36ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:05.301999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24785edc-9c8f-4edf-a75f-a301657b36ef None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:05.302285 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24785edc-9c8f-4edf-a75f-a301657b36ef None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:05.302611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24785edc-9c8f-4edf-a75f-a301657b36ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:05.310503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:05.314765 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc931e4-a0a4-4818-ac74-54b178ee543c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:05.316803 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24785edc-9c8f-4edf-a75f-a301657b36ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-24785edc-9c8f-4edf-a75f-a301657b36ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:05.321753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:05.322770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:05.328906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:05.328906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:05.328906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:05.328906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:05.328906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:05.328906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:05.328906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:05.328906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:05.328906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:05.328906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:05.328906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:05.328906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:05.328906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 32/71] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:59:05.328906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ddbba-3337-444c-9f3a-7e21f5bfd74c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:05.329902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ddbba-3337-444c-9f3a-7e21f5bfd74c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:05.329902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ddbba-3337-444c-9f3a-7e21f5bfd74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:05.347364 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d62ddbba-3337-444c-9f3a-7e21f5bfd74c None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:05.348730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d62ddbba-3337-444c-9f3a-7e21f5bfd74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:05.360314 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87293dd-bad0-4d41-a59c-23fd1210cd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:05.361157 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87293dd-bad0-4d41-a59c-23fd1210cd53 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:05.361573 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87293dd-bad0-4d41-a59c-23fd1210cd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:05.361980 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87293dd-bad0-4d41-a59c-23fd1210cd53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:05.373188 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87293dd-bad0-4d41-a59c-23fd1210cd53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-b87293dd-bad0-4d41-a59c-23fd1210cd53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:05.378311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:05.379287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:05.380175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:05.380175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:05.380175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:05.380175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:05.380175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:05.380175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:05.380175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:05.380175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:05.380175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:05.380175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:05.380175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:05.380175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:05.382138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 40/72] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 05:59:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:05.383082 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87293dd-bad0-4d41-a59c-23fd1210cd53 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:05.408404 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:05.425800 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 33/73] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:05.426875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ddbba-3337-444c-9f3a-7e21f5bfd74c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:05.511931 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87293dd-bad0-4d41-a59c-23fd1210cd53 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:05.550051 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ded8d4-db68-4eea-b0d8-1f460d78a16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:05.550688 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ded8d4-db68-4eea-b0d8-1f460d78a16c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:05.550792 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ddbba-3337-444c-9f3a-7e21f5bfd74c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:05.551192 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ded8d4-db68-4eea-b0d8-1f460d78a16c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:05.551192 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ded8d4-db68-4eea-b0d8-1f460d78a16c None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:05.551869 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ded8d4-db68-4eea-b0d8-1f460d78a16c None admin] RBAC: Adding query filter params (name=member) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:05.552420 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ded8d4-db68-4eea-b0d8-1f460d78a16c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:05.554554 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ded8d4-db68-4eea-b0d8-1f460d78a16c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:05.566852 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 41/74] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:05.567567 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ded8d4-db68-4eea-b0d8-1f460d78a16c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:05.585271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:05.595549 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b01fd5-d3e3-4504-b51e-c29a5df461ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:05.596167 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b01fd5-d3e3-4504-b51e-c29a5df461ea None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:05.596453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b01fd5-d3e3-4504-b51e-c29a5df461ea None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:05.596820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b01fd5-d3e3-4504-b51e-c29a5df461ea None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:05.597396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b01fd5-d3e3-4504-b51e-c29a5df461ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:05.597699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b01fd5-d3e3-4504-b51e-c29a5df461ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:05.599111 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b01fd5-d3e3-4504-b51e-c29a5df461ea None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:05.611724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 34/75] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:05.616803 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b01fd5-d3e3-4504-b51e-c29a5df461ea None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:05.666558 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:05.686693 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ded8d4-db68-4eea-b0d8-1f460d78a16c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:05.723719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f238e74-2d02-4704-b52b-f99107611359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:05.724715 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f238e74-2d02-4704-b52b-f99107611359 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:05.725003 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f238e74-2d02-4704-b52b-f99107611359 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:05.725279 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f238e74-2d02-4704-b52b-f99107611359 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:05.726929 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b01fd5-d3e3-4504-b51e-c29a5df461ea None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:05.735985 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f238e74-2d02-4704-b52b-f99107611359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-8f238e74-2d02-4704-b52b-f99107611359 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:05.741148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:05.742179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:05.743413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:05.743413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:05.743413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:05.743413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:05.743413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:05.743413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:05.743413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:05.743413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:05.743413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:05.743413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:05.743413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:05.743413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:05.743413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 42/76] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:05.743413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f238e74-2d02-4704-b52b-f99107611359 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:05.766877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7316c-2ca1-4ad4-9f73-f3bc60b5f89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:05.767514 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7316c-2ca1-4ad4-9f73-f3bc60b5f89d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:05.767795 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7316c-2ca1-4ad4-9f73-f3bc60b5f89d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:05.768067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7316c-2ca1-4ad4-9f73-f3bc60b5f89d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:05.778083 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7316c-2ca1-4ad4-9f73-f3bc60b5f89d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-e6c7316c-2ca1-4ad4-9f73-f3bc60b5f89d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:05.783086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:05.783921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:05.784712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:05.784712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:05.784712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:05.784712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:05.784712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:05.784712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:05.784712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:05.784712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:05.784712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:05.784712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:05.784712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:05.784712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:05.784712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 35/77] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 05:59:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:05.786847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7316c-2ca1-4ad4-9f73-f3bc60b5f89d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:05.840402 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:05.846710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f238e74-2d02-4704-b52b-f99107611359 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:05.880367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ec3e6-03aa-4eab-9474-8b4658ce8777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:05.880988 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ec3e6-03aa-4eab-9474-8b4658ce8777 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:05.881261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ec3e6-03aa-4eab-9474-8b4658ce8777 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:05.881530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ec3e6-03aa-4eab-9474-8b4658ce8777 None admin] PATH_INFO: `/v3/users/20bcd5c59c07488b83a3391be92d86f5` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:05.891674 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7316c-2ca1-4ad4-9f73-f3bc60b5f89d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:05.893271 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:05.896896 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ec3e6-03aa-4eab-9474-8b4658ce8777 None admin] RBAC: Authorizing `identity:get_user(user_id=20bcd5c59c07488b83a3391be92d86f5)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:05.898667 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ec3e6-03aa-4eab-9474-8b4658ce8777 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:05.900526 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 43/78] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:05 2022] GET /identity/v3/users/20bcd5c59c07488b83a3391be92d86f5 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:05.901312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ec3e6-03aa-4eab-9474-8b4658ce8777 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:05.933057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd39813-7684-438c-b9d7-dc75dcd68c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:05.933704 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd39813-7684-438c-b9d7-dc75dcd68c15 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:05.933990 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd39813-7684-438c-b9d7-dc75dcd68c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:05.934269 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd39813-7684-438c-b9d7-dc75dcd68c15 None admin] PATH_INFO: `/v3/users/c922803daa44438cbb8185e3898e3dc9` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:05.935464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd39813-7684-438c-b9d7-dc75dcd68c15 None admin] RBAC: Authorizing `identity:get_user(user_id=c922803daa44438cbb8185e3898e3dc9)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:05.936926 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd39813-7684-438c-b9d7-dc75dcd68c15 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:05.939623 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 36/79] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:05 2022] GET /identity/v3/users/c922803daa44438cbb8185e3898e3dc9 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:05.940427 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd39813-7684-438c-b9d7-dc75dcd68c15 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:06.006196 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ec3e6-03aa-4eab-9474-8b4658ce8777 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:06.039520 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18af96-2e5b-43e6-a0d6-f3fa47b8c2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:06.040145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18af96-2e5b-43e6-a0d6-f3fa47b8c2be None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:06.040426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18af96-2e5b-43e6-a0d6-f3fa47b8c2be None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:06.040700 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18af96-2e5b-43e6-a0d6-f3fa47b8c2be None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:06.041807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18af96-2e5b-43e6-a0d6-f3fa47b8c2be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:06.043284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18af96-2e5b-43e6-a0d6-f3fa47b8c2be None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:06.044538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd39813-7684-438c-b9d7-dc75dcd68c15 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:06.044854 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 44/80] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:59:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:06.045622 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18af96-2e5b-43e6-a0d6-f3fa47b8c2be None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:06.068879 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:06.089219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f03704-8384-4951-b835-46baf4cb14a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:06.090152 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f03704-8384-4951-b835-46baf4cb14a0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:06.090493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f03704-8384-4951-b835-46baf4cb14a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:06.090759 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f03704-8384-4951-b835-46baf4cb14a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:06.091433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f03704-8384-4951-b835-46baf4cb14a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:06.091739 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f03704-8384-4951-b835-46baf4cb14a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:06.093124 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f03704-8384-4951-b835-46baf4cb14a0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:06.101241 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 37/81] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:06.102056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f03704-8384-4951-b835-46baf4cb14a0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:06.117393 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:06.153381 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18af96-2e5b-43e6-a0d6-f3fa47b8c2be None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:06.187985 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791633e4-769f-4902-96fa-e3d7b42a82ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:06.188667 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791633e4-769f-4902-96fa-e3d7b42a82ce None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:06.188953 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791633e4-769f-4902-96fa-e3d7b42a82ce None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:06.189230 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791633e4-769f-4902-96fa-e3d7b42a82ce None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:06.189900 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791633e4-769f-4902-96fa-e3d7b42a82ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:06.190204 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791633e4-769f-4902-96fa-e3d7b42a82ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:06.191652 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791633e4-769f-4902-96fa-e3d7b42a82ce None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:06.199693 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 45/82] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 05:59:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:06.199693 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791633e4-769f-4902-96fa-e3d7b42a82ce None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:06.206867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f03704-8384-4951-b835-46baf4cb14a0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:06.241970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d778c9-cd19-424b-abd6-1b55f2d3ac30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:06.242612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d778c9-cd19-424b-abd6-1b55f2d3ac30 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:06.243147 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d778c9-cd19-424b-abd6-1b55f2d3ac30 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:06.243419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d778c9-cd19-424b-abd6-1b55f2d3ac30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:06.291974 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:06.305106 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791633e4-769f-4902-96fa-e3d7b42a82ce None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:06.313660 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d778c9-cd19-424b-abd6-1b55f2d3ac30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:06.315186 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d778c9-cd19-424b-abd6-1b55f2d3ac30 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:06.333562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:06.340258 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f135c0-644e-4452-b6fb-8c8efd2de663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:06.341107 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f135c0-644e-4452-b6fb-8c8efd2de663 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:06.341339 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f135c0-644e-4452-b6fb-8c8efd2de663 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:06.341633 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f135c0-644e-4452-b6fb-8c8efd2de663 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:06.412162 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f135c0-644e-4452-b6fb-8c8efd2de663 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:06.413655 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f135c0-644e-4452-b6fb-8c8efd2de663 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:06.415815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 38/83] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:06.416658 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d778c9-cd19-424b-abd6-1b55f2d3ac30 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:06.496228 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:06.513971 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 46/84] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:06.514891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f135c0-644e-4452-b6fb-8c8efd2de663 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:06.517232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d778c9-cd19-424b-abd6-1b55f2d3ac30 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:06.545953 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:06.551346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7b3b1-baeb-400f-a538-a1c30aa747be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:06.551964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7b3b1-baeb-400f-a538-a1c30aa747be None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:06.552243 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7b3b1-baeb-400f-a538-a1c30aa747be None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:06.552513 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7b3b1-baeb-400f-a538-a1c30aa747be None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:06.553158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7b3b1-baeb-400f-a538-a1c30aa747be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:06.554670 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7b3b1-baeb-400f-a538-a1c30aa747be None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:06.582036 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 39/85] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 05:59:06 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 05:59:06.587605 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7b3b1-baeb-400f-a538-a1c30aa747be None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:06.626628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f135c0-644e-4452-b6fb-8c8efd2de663 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:06.671276 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d6b39-fd8b-48d0-9038-15fbe147b4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:06.672013 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d6b39-fd8b-48d0-9038-15fbe147b4ef None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:06.672296 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d6b39-fd8b-48d0-9038-15fbe147b4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:06.672571 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d6b39-fd8b-48d0-9038-15fbe147b4ef None admin] PATH_INFO: `/v3/users/20bcd5c59c07488b83a3391be92d86f5` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:06.673948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d6b39-fd8b-48d0-9038-15fbe147b4ef None admin] RBAC: Authorizing `identity:get_user(user_id=20bcd5c59c07488b83a3391be92d86f5)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:06.675488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d6b39-fd8b-48d0-9038-15fbe147b4ef None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:06.681981 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 47/86] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:06 2022] GET /identity/v3/users/20bcd5c59c07488b83a3391be92d86f5 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:06.681981 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d6b39-fd8b-48d0-9038-15fbe147b4ef None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:06.712145 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7b3b1-baeb-400f-a538-a1c30aa747be None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:06.759366 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0b8b7-6db5-43e1-ac17-19624422d4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:06.760016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0b8b7-6db5-43e1-ac17-19624422d4eb None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:06.760388 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0b8b7-6db5-43e1-ac17-19624422d4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:06.760584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0b8b7-6db5-43e1-ac17-19624422d4eb None admin] PATH_INFO: `/v3/users/20bcd5c59c07488b83a3391be92d86f5` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:06.761806 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0b8b7-6db5-43e1-ac17-19624422d4eb None admin] RBAC: Authorizing `identity:get_user(user_id=20bcd5c59c07488b83a3391be92d86f5)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:06.763721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0b8b7-6db5-43e1-ac17-19624422d4eb None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:06.766446 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 40/87] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:06 2022] GET /identity/v3/users/20bcd5c59c07488b83a3391be92d86f5 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:06.767481 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0b8b7-6db5-43e1-ac17-19624422d4eb None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:06.772101 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:06.821828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:06.825576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d6b39-fd8b-48d0-9038-15fbe147b4ef None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:06.877125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91e1dc-b684-43bc-9cbc-8fe7b49a40f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:06.878143 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91e1dc-b684-43bc-9cbc-8fe7b49a40f5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:06.880267 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91e1dc-b684-43bc-9cbc-8fe7b49a40f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:06.880762 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91e1dc-b684-43bc-9cbc-8fe7b49a40f5 None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:06.882188 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91e1dc-b684-43bc-9cbc-8fe7b49a40f5 None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:06.883932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91e1dc-b684-43bc-9cbc-8fe7b49a40f5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:06.885810 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 48/88] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:06 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:06.886790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91e1dc-b684-43bc-9cbc-8fe7b49a40f5 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:06.894173 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0b8b7-6db5-43e1-ac17-19624422d4eb None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:06.939559 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc41a8-381b-4171-8a5b-15a8b8ed5ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:06.940213 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc41a8-381b-4171-8a5b-15a8b8ed5ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:06.940492 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc41a8-381b-4171-8a5b-15a8b8ed5ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:06.940764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc41a8-381b-4171-8a5b-15a8b8ed5ab7 None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:06.941944 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc41a8-381b-4171-8a5b-15a8b8ed5ab7 None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:06.943437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc41a8-381b-4171-8a5b-15a8b8ed5ab7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:06.945106 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 41/89] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:06 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:06.946031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc41a8-381b-4171-8a5b-15a8b8ed5ab7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:07.028064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91e1dc-b684-43bc-9cbc-8fe7b49a40f5 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:07.034961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:07.071170 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282ee56-a45f-4439-b546-114f9c82d9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:07.072653 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282ee56-a45f-4439-b546-114f9c82d9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:07.073080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282ee56-a45f-4439-b546-114f9c82d9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:07.073832 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282ee56-a45f-4439-b546-114f9c82d9c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:07.075570 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc41a8-381b-4171-8a5b-15a8b8ed5ab7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:07.113425 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925e0a1-f7ec-4512-aae0-0253cd8e06ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:07.114186 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925e0a1-f7ec-4512-aae0-0253cd8e06ea None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:07.114588 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925e0a1-f7ec-4512-aae0-0253cd8e06ea None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:07.114962 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925e0a1-f7ec-4512-aae0-0253cd8e06ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:07.125887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:07.175843 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282ee56-a45f-4439-b546-114f9c82d9c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:07.178215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282ee56-a45f-4439-b546-114f9c82d9c4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:07.196596 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925e0a1-f7ec-4512-aae0-0253cd8e06ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:07.198994 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925e0a1-f7ec-4512-aae0-0253cd8e06ea None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:07.286999 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 49/90] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:07.287183 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:07.292289 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282ee56-a45f-4439-b546-114f9c82d9c4 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:07.317201 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 42/91] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:07.318550 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925e0a1-f7ec-4512-aae0-0253cd8e06ea None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:07.365310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:07.402785 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282ee56-a45f-4439-b546-114f9c82d9c4 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:07.429471 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925e0a1-f7ec-4512-aae0-0253cd8e06ea None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:07.440267 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4780e17-d5dd-454b-9901-f20230c1aaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:07.440981 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4780e17-d5dd-454b-9901-f20230c1aaf4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:07.441376 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4780e17-d5dd-454b-9901-f20230c1aaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:07.441745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4780e17-d5dd-454b-9901-f20230c1aaf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:07.442718 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4780e17-d5dd-454b-9901-f20230c1aaf4 None admin] RBAC: Adding query filter params (role.id=a2f35dad17e94553b63d07f8b8d8f1a4, scope.project.id=bca76726c98f48f482c676027eb393d2, user.id=20bcd5c59c07488b83a3391be92d86f5) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:07.443120 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4780e17-d5dd-454b-9901-f20230c1aaf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:07.444681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4780e17-d5dd-454b-9901-f20230c1aaf4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:07.451418 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 50/92] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:07 2022] GET /identity/v3/role_assignments?user.id=20bcd5c59c07488b83a3391be92d86f5&scope.project.id=bca76726c98f48f482c676027eb393d2&role.id=a2f35dad17e94553b63d07f8b8d8f1a4 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:07.452404 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4780e17-d5dd-454b-9901-f20230c1aaf4 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:07.470032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d2c31-efc9-41e2-a3a4-cafba79cc66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:07.471021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d2c31-efc9-41e2-a3a4-cafba79cc66c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:07.471531 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d2c31-efc9-41e2-a3a4-cafba79cc66c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:07.472010 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d2c31-efc9-41e2-a3a4-cafba79cc66c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:07.472955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d2c31-efc9-41e2-a3a4-cafba79cc66c None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=bca76726c98f48f482c676027eb393d2, user.id=c922803daa44438cbb8185e3898e3dc9) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:07.473469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d2c31-efc9-41e2-a3a4-cafba79cc66c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:07.475405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d2c31-efc9-41e2-a3a4-cafba79cc66c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:07.485505 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 43/93] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:07 2022] GET /identity/v3/role_assignments?user.id=c922803daa44438cbb8185e3898e3dc9&scope.project.id=bca76726c98f48f482c676027eb393d2&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:07.486746 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d2c31-efc9-41e2-a3a4-cafba79cc66c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:07.530724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:07.573849 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4780e17-d5dd-454b-9901-f20230c1aaf4 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:07.606938 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:07.613854 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ff109-c872-4aa4-88fc-412adf5e19a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:07.614983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ff109-c872-4aa4-88fc-412adf5e19a6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:07.616875 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ff109-c872-4aa4-88fc-412adf5e19a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:07.617404 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ff109-c872-4aa4-88fc-412adf5e19a6 None admin] PATH_INFO: `/v3/projects/5d640d7d4d914dd4b88d4203e8a0f0e0` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:07.619199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ff109-c872-4aa4-88fc-412adf5e19a6 None admin] RBAC: Authorizing `identity:get_project(project_id=5d640d7d4d914dd4b88d4203e8a0f0e0)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:07.620819 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ff109-c872-4aa4-88fc-412adf5e19a6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:07.622037 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d2c31-efc9-41e2-a3a4-cafba79cc66c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:07.622803 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 51/94] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:07 2022] GET /identity/v3/projects/5d640d7d4d914dd4b88d4203e8a0f0e0 => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:07.630024 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ff109-c872-4aa4-88fc-412adf5e19a6 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:07.664075 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9f66e-bb2a-4f4a-a189-f2cb2fb4f800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:07.665009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9f66e-bb2a-4f4a-a189-f2cb2fb4f800 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:07.665456 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9f66e-bb2a-4f4a-a189-f2cb2fb4f800 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:07.666021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9f66e-bb2a-4f4a-a189-f2cb2fb4f800 None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:07.667528 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9f66e-bb2a-4f4a-a189-f2cb2fb4f800 None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:07.669172 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9f66e-bb2a-4f4a-a189-f2cb2fb4f800 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:07.671250 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 44/95] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:07 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:07.679659 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9f66e-bb2a-4f4a-a189-f2cb2fb4f800 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:07.760550 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ff109-c872-4aa4-88fc-412adf5e19a6 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:07.798706 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f759d-3bcb-4365-9340-22569477b22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:07.799485 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f759d-3bcb-4365-9340-22569477b22f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:07.799858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f759d-3bcb-4365-9340-22569477b22f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:07.800254 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f759d-3bcb-4365-9340-22569477b22f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:07.806540 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f759d-3bcb-4365-9340-22569477b22f None admin] RBAC: Adding query filter params (role.id=a2f35dad17e94553b63d07f8b8d8f1a4, scope.project.id=5d640d7d4d914dd4b88d4203e8a0f0e0, user.id=20bcd5c59c07488b83a3391be92d86f5) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:07.806540 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f759d-3bcb-4365-9340-22569477b22f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:07.806540 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f759d-3bcb-4365-9340-22569477b22f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:07.809298 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 52/96] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:07 2022] GET /identity/v3/role_assignments?user.id=20bcd5c59c07488b83a3391be92d86f5&scope.project.id=5d640d7d4d914dd4b88d4203e8a0f0e0&role.id=a2f35dad17e94553b63d07f8b8d8f1a4 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:07.841241 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9f66e-bb2a-4f4a-a189-f2cb2fb4f800 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:07.846399 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:07.890135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41042bcd-4b9b-454c-a31a-0d1cefe9de83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:07.891023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41042bcd-4b9b-454c-a31a-0d1cefe9de83 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:07.891303 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41042bcd-4b9b-454c-a31a-0d1cefe9de83 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:07.891626 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41042bcd-4b9b-454c-a31a-0d1cefe9de83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:07.892379 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41042bcd-4b9b-454c-a31a-0d1cefe9de83 None admin] RBAC: Adding query filter params (role.id=f47755365f24477f9846225c9e91de2d, scope.project.id=bca76726c98f48f482c676027eb393d2, user.id=20bcd5c59c07488b83a3391be92d86f5) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:07.892684 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41042bcd-4b9b-454c-a31a-0d1cefe9de83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:07.894385 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41042bcd-4b9b-454c-a31a-0d1cefe9de83 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:07.901057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 45/97] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:07 2022] GET /identity/v3/role_assignments?user.id=20bcd5c59c07488b83a3391be92d86f5&scope.project.id=bca76726c98f48f482c676027eb393d2&role.id=f47755365f24477f9846225c9e91de2d => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:09.325880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46ab89-55e0-4577-a602-71742de00992 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:09.326388 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46ab89-55e0-4577-a602-71742de00992 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:09.326461 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46ab89-55e0-4577-a602-71742de00992 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:09.327986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 53/98] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:09.342288 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e543d8f-93a6-43ad-93bb-b9b0ac5e146f None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:09.342766 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e543d8f-93a6-43ad-93bb-b9b0ac5e146f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:09.342965 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e543d8f-93a6-43ad-93bb-b9b0ac5e146f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:09.370881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1e543d8f-93a6-43ad-93bb-b9b0ac5e146f None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:09.372097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1e543d8f-93a6-43ad-93bb-b9b0ac5e146f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:09.374006 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f162a6-f44e-4458-8922-4f047a3800ef None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:09.374283 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f162a6-f44e-4458-8922-4f047a3800ef None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:09.375088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f162a6-f44e-4458-8922-4f047a3800ef None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:09.376591 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 54/99] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:59:09.381834 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48250033-85d3-40f5-b166-884d4dbba301 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:09.382117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48250033-85d3-40f5-b166-884d4dbba301 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:09.382393 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48250033-85d3-40f5-b166-884d4dbba301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:09.413582 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-48250033-85d3-40f5-b166-884d4dbba301 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:09.414827 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-48250033-85d3-40f5-b166-884d4dbba301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:09.526909 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:09.537107 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 55/100] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:59:09.543826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 46/101] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:09.873906 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48250033-85d3-40f5-b166-884d4dbba301 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:10.002897 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48250033-85d3-40f5-b166-884d4dbba301 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:10.037637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e751c5-7788-46d5-ab77-9a2b8aa18765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:10.038273 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e751c5-7788-46d5-ab77-9a2b8aa18765 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:10.038569 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e751c5-7788-46d5-ab77-9a2b8aa18765 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:10.038881 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e751c5-7788-46d5-ab77-9a2b8aa18765 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:10.049666 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e751c5-7788-46d5-ab77-9a2b8aa18765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-76e751c5-7788-46d5-ab77-9a2b8aa18765 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:10.054469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:10.055306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:10.056266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:10.056266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:10.056266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:10.056266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:10.056266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:10.056266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:10.056266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:10.056266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:10.056266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:10.056266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:10.056266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:10.056266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:10.056266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 56/102] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:10.067639 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e543d8f-93a6-43ad-93bb-b9b0ac5e146f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:10.095156 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e751c5-7788-46d5-ab77-9a2b8aa18765 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:10.172661 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:10.245253 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e751c5-7788-46d5-ab77-9a2b8aa18765 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:10.280191 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:10.287138 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e543d8f-93a6-43ad-93bb-b9b0ac5e146f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:10.288106 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bdf34-727d-4d20-8a4b-14d23722c7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:10.288804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bdf34-727d-4d20-8a4b-14d23722c7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:10.290102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bdf34-727d-4d20-8a4b-14d23722c7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:10.290433 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bdf34-727d-4d20-8a4b-14d23722c7f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:10.291789 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bdf34-727d-4d20-8a4b-14d23722c7f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:10.293345 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bdf34-727d-4d20-8a4b-14d23722c7f5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:10.295027 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 57/103] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:59:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:10.300708 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bdf34-727d-4d20-8a4b-14d23722c7f5 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:10.384303 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093035e-ded3-40bf-bf05-7b0e565eb007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:10.384957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093035e-ded3-40bf-bf05-7b0e565eb007 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:10.385227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093035e-ded3-40bf-bf05-7b0e565eb007 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:10.385496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093035e-ded3-40bf-bf05-7b0e565eb007 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:10.386182 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093035e-ded3-40bf-bf05-7b0e565eb007 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:10.386492 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093035e-ded3-40bf-bf05-7b0e565eb007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:10.387947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093035e-ded3-40bf-bf05-7b0e565eb007 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:10.407489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bdf34-727d-4d20-8a4b-14d23722c7f5 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:10.409055 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 47/104] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:10.410132 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57fc4c-15c1-46ce-a17b-5345cff81559 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:10.410423 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57fc4c-15c1-46ce-a17b-5345cff81559 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:10.410731 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57fc4c-15c1-46ce-a17b-5345cff81559 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:10.411968 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 48/105] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:10.419905 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452007fd-323e-4caa-b306-af36d33620c0 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:10.420198 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452007fd-323e-4caa-b306-af36d33620c0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:10.420458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452007fd-323e-4caa-b306-af36d33620c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:10.439289 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:10.444830 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79223883-b5a5-4818-85fe-73a7d8715548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:10.445459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79223883-b5a5-4818-85fe-73a7d8715548 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:10.445741 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79223883-b5a5-4818-85fe-73a7d8715548 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:10.446015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79223883-b5a5-4818-85fe-73a7d8715548 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:10.446711 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79223883-b5a5-4818-85fe-73a7d8715548 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:10.448688 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79223883-b5a5-4818-85fe-73a7d8715548 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:10.467821 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-452007fd-323e-4caa-b306-af36d33620c0 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:10.469513 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-452007fd-323e-4caa-b306-af36d33620c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:10.509339 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 58/106] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 23 05:59:10 2022] POST /identity/v3/projects => generated 302 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 05:59:10.510288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79223883-b5a5-4818-85fe-73a7d8715548 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:10.589418 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 49/107] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:10.621588 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3002fa-1115-4c12-b403-b97dba7e4743 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:10.621936 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3002fa-1115-4c12-b403-b97dba7e4743 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:10.622158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3002fa-1115-4c12-b403-b97dba7e4743 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:10.623611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 50/108] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:59:10.629843 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367cc77-5a78-461c-83b7-8d7493d31b03 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:10.630009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367cc77-5a78-461c-83b7-8d7493d31b03 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:10.630317 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367cc77-5a78-461c-83b7-8d7493d31b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:10.648712 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79223883-b5a5-4818-85fe-73a7d8715548 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:10.650546 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1367cc77-5a78-461c-83b7-8d7493d31b03 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:10.651882 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1367cc77-5a78-461c-83b7-8d7493d31b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:10.698446 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a2084-cf34-4f5a-883e-ba06688c6dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:10.700247 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a2084-cf34-4f5a-883e-ba06688c6dff None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:10.700247 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a2084-cf34-4f5a-883e-ba06688c6dff None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:10.704349 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a2084-cf34-4f5a-883e-ba06688c6dff None admin] PATH_INFO: `/v3/users/b53182f546814164abd49c1611eca319` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:10.746336 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a2084-cf34-4f5a-883e-ba06688c6dff None admin] RBAC: Authorizing `identity:get_user(user_id=b53182f546814164abd49c1611eca319)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:10.749575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a2084-cf34-4f5a-883e-ba06688c6dff None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:10.752108 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 59/109] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:10 2022] GET /identity/v3/users/b53182f546814164abd49c1611eca319 => generated 289 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:10.760063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a2084-cf34-4f5a-883e-ba06688c6dff None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:10.782946 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 51/110] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:10.810052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:10.907776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a2084-cf34-4f5a-883e-ba06688c6dff None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:10.962535 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06069e31-6588-495c-b5fe-70851890785d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:10.962535 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06069e31-6588-495c-b5fe-70851890785d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:10.962535 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06069e31-6588-495c-b5fe-70851890785d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:10.962535 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06069e31-6588-495c-b5fe-70851890785d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:11.033359 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1a352-a1b0-49c5-8850-b547f0c55ff3 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:11.033709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1a352-a1b0-49c5-8850-b547f0c55ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:11.033926 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1a352-a1b0-49c5-8850-b547f0c55ff3 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:11.035402 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 52/111] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:11.041021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b34bb8-f4a0-4238-a56a-1610dbf67947 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:11.041312 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b34bb8-f4a0-4238-a56a-1610dbf67947 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:11.041593 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b34bb8-f4a0-4238-a56a-1610dbf67947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:11.058131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06069e31-6588-495c-b5fe-70851890785d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:11.059666 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06069e31-6588-495c-b5fe-70851890785d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:11.066839 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-96b34bb8-f4a0-4238-a56a-1610dbf67947 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:11.068105 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-96b34bb8-f4a0-4238-a56a-1610dbf67947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:11.088019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:11.101499 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:11.160769 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 53/112] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:11.161639 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b34bb8-f4a0-4238-a56a-1610dbf67947 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:11.200256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 60/113] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:11.202981 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca08b2f-8917-496d-920a-036d790d2f49 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:11.203504 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca08b2f-8917-496d-920a-036d790d2f49 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:11.203975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca08b2f-8917-496d-920a-036d790d2f49 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:11.205623 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 61/114] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:11.206559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca08b2f-8917-496d-920a-036d790d2f49 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:11.309836 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b34bb8-f4a0-4238-a56a-1610dbf67947 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:11.327339 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca08b2f-8917-496d-920a-036d790d2f49 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:11.357016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068d8b5-a4d4-43ca-9e48-0e904cff4572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:11.357905 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068d8b5-a4d4-43ca-9e48-0e904cff4572 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:11.358592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068d8b5-a4d4-43ca-9e48-0e904cff4572 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:11.358957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068d8b5-a4d4-43ca-9e48-0e904cff4572 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:11.362951 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:11.373660 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09adeb3-0ad8-41dd-a6c5-ec30ad20e5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:11.373839 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068d8b5-a4d4-43ca-9e48-0e904cff4572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:11.374614 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09adeb3-0ad8-41dd-a6c5-ec30ad20e5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:11.375128 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09adeb3-0ad8-41dd-a6c5-ec30ad20e5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:11.375535 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09adeb3-0ad8-41dd-a6c5-ec30ad20e5b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-c068d8b5-a4d4-43ca-9e48-0e904cff4572 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:11.378886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:11.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:11.380426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:11.380426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:11.380426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:11.380426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:11.380426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:11.380426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:11.380426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:11.380426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:11.380426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:11.380426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:11.380426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:11.380426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:11.380426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 54/115] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 05:59:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:11.381026 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068d8b5-a4d4-43ca-9e48-0e904cff4572 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:11.392863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09adeb3-0ad8-41dd-a6c5-ec30ad20e5b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:11.399034 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-f09adeb3-0ad8-41dd-a6c5-ec30ad20e5b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:11.400113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:11.401530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:11.403654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:11.403654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:11.403654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:11.403654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:11.403654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:11.403654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:11.403654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:11.403654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:11.403654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:11.403654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:11.403654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:11.403654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:11.403654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 62/116] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:11.403654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b61a97-4ca5-4822-a1b7-4a4854f941ef None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:11.404584 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b61a97-4ca5-4822-a1b7-4a4854f941ef None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:11.404584 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b61a97-4ca5-4822-a1b7-4a4854f941ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:11.431737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b8b61a97-4ca5-4822-a1b7-4a4854f941ef None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:11.434269 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b8b61a97-4ca5-4822-a1b7-4a4854f941ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:11.508875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068d8b5-a4d4-43ca-9e48-0e904cff4572 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:11.511102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 63/117] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:11.511784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b61a97-4ca5-4822-a1b7-4a4854f941ef None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:11.547564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278298c6-d018-41fe-be15-61082fea08ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:11.548477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278298c6-d018-41fe-be15-61082fea08ab None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:11.548924 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278298c6-d018-41fe-be15-61082fea08ab None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:11.549383 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278298c6-d018-41fe-be15-61082fea08ab None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:11.550909 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278298c6-d018-41fe-be15-61082fea08ab None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:11.552516 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278298c6-d018-41fe-be15-61082fea08ab None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:11.554614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 55/118] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:11 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:11.555549 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278298c6-d018-41fe-be15-61082fea08ab None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:11.654016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b61a97-4ca5-4822-a1b7-4a4854f941ef None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:11.661726 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:11.685436 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:11.690081 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278298c6-d018-41fe-be15-61082fea08ab None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:11.707931 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a62dc-f6c7-496c-9229-acf08128b50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:11.709141 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a62dc-f6c7-496c-9229-acf08128b50b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:11.709617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a62dc-f6c7-496c-9229-acf08128b50b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:11.710078 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a62dc-f6c7-496c-9229-acf08128b50b None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:11.710958 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a62dc-f6c7-496c-9229-acf08128b50b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:11.711955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a62dc-f6c7-496c-9229-acf08128b50b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:11.713835 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a62dc-f6c7-496c-9229-acf08128b50b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:11.727059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 64/119] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:11.727211 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4a8b5-8c64-4348-a5fb-4a8935125918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:11.728221 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a62dc-f6c7-496c-9229-acf08128b50b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:11.732717 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4a8b5-8c64-4348-a5fb-4a8935125918 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:11.733076 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4a8b5-8c64-4348-a5fb-4a8935125918 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:11.733428 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4a8b5-8c64-4348-a5fb-4a8935125918 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:11.734875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4a8b5-8c64-4348-a5fb-4a8935125918 None admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:11.735298 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4a8b5-8c64-4348-a5fb-4a8935125918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:11.736775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4a8b5-8c64-4348-a5fb-4a8935125918 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:11.751344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 56/120] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:11.752342 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4a8b5-8c64-4348-a5fb-4a8935125918 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:11.846998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a62dc-f6c7-496c-9229-acf08128b50b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:11.867372 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4a8b5-8c64-4348-a5fb-4a8935125918 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:11.884978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395381d-f3fd-45d7-9fb1-7b33b7ac767d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:11.885669 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395381d-f3fd-45d7-9fb1-7b33b7ac767d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:11.885952 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395381d-f3fd-45d7-9fb1-7b33b7ac767d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:11.886278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395381d-f3fd-45d7-9fb1-7b33b7ac767d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:11.897313 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395381d-f3fd-45d7-9fb1-7b33b7ac767d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-d395381d-f3fd-45d7-9fb1-7b33b7ac767d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:11.902080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:11.903063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:11.903813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:11.903813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:11.903813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:11.903813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:11.903813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:11.903813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:11.903813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:11.903813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:11.903813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:11.903813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:11.903813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:11.903813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:11.903813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 65/121] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:11.904438 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395381d-f3fd-45d7-9fb1-7b33b7ac767d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:11.907728 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924fe8b-8483-4e05-b78d-2270920593c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:11.908577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924fe8b-8483-4e05-b78d-2270920593c8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:11.908862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924fe8b-8483-4e05-b78d-2270920593c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:11.909137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924fe8b-8483-4e05-b78d-2270920593c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:11.909880 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4924fe8b-8483-4e05-b78d-2270920593c8 None admin] RBAC: Adding query filter params (role.id=86e796f30478458d99f1d139c853a1db, scope.project.id=bca76726c98f48f482c676027eb393d2, user.id=b53182f546814164abd49c1611eca319) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:11.910190 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4924fe8b-8483-4e05-b78d-2270920593c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:11.911839 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4924fe8b-8483-4e05-b78d-2270920593c8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:11.919262 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 57/122] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:11 2022] GET /identity/v3/role_assignments?user.id=b53182f546814164abd49c1611eca319&scope.project.id=bca76726c98f48f482c676027eb393d2&role.id=86e796f30478458d99f1d139c853a1db => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:11.920170 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924fe8b-8483-4e05-b78d-2270920593c8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:11.926622 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:11.979272 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:12.060436 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395381d-f3fd-45d7-9fb1-7b33b7ac767d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:12.076020 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924fe8b-8483-4e05-b78d-2270920593c8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:12.113082 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51b7ab-8bcc-4271-859d-0d10c938ae1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:12.113774 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51b7ab-8bcc-4271-859d-0d10c938ae1b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:12.114061 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51b7ab-8bcc-4271-859d-0d10c938ae1b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:12.114336 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51b7ab-8bcc-4271-859d-0d10c938ae1b None admin] PATH_INFO: `/v3/users/c922803daa44438cbb8185e3898e3dc9` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:12.114923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da8c63-7c75-4c12-8d3a-1a4919ac2423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:12.115534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51b7ab-8bcc-4271-859d-0d10c938ae1b None admin] RBAC: Authorizing `identity:get_user(user_id=c922803daa44438cbb8185e3898e3dc9)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:12.116075 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da8c63-7c75-4c12-8d3a-1a4919ac2423 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:12.116542 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da8c63-7c75-4c12-8d3a-1a4919ac2423 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:12.117002 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51b7ab-8bcc-4271-859d-0d10c938ae1b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:12.117113 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da8c63-7c75-4c12-8d3a-1a4919ac2423 None admin] PATH_INFO: `/v3/users/20bcd5c59c07488b83a3391be92d86f5` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:12.118730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da8c63-7c75-4c12-8d3a-1a4919ac2423 None admin] RBAC: Authorizing `identity:get_user(user_id=20bcd5c59c07488b83a3391be92d86f5)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:12.119017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 66/123] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:11 2022] GET /identity/v3/users/c922803daa44438cbb8185e3898e3dc9 => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:12.119868 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51b7ab-8bcc-4271-859d-0d10c938ae1b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:12.120939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da8c63-7c75-4c12-8d3a-1a4919ac2423 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:12.122783 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 58/124] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:11 2022] GET /identity/v3/users/20bcd5c59c07488b83a3391be92d86f5 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:12.123755 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da8c63-7c75-4c12-8d3a-1a4919ac2423 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:12.204600 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:12.249028 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da8c63-7c75-4c12-8d3a-1a4919ac2423 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:12.258413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51b7ab-8bcc-4271-859d-0d10c938ae1b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:12.271654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:12.297774 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f961d25-d522-4874-a5e9-00ffc9dcb217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:12.297996 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e72d7a-5a57-4150-bd56-1bf093d31eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:12.298395 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e72d7a-5a57-4150-bd56-1bf093d31eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:12.298708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e72d7a-5a57-4150-bd56-1bf093d31eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:12.298982 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e72d7a-5a57-4150-bd56-1bf093d31eb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:12.299535 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f961d25-d522-4874-a5e9-00ffc9dcb217 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:12.299633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e72d7a-5a57-4150-bd56-1bf093d31eb7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:12.299946 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e72d7a-5a57-4150-bd56-1bf093d31eb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:12.300277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f961d25-d522-4874-a5e9-00ffc9dcb217 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:12.300742 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f961d25-d522-4874-a5e9-00ffc9dcb217 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:12.301319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e72d7a-5a57-4150-bd56-1bf093d31eb7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:12.312652 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f961d25-d522-4874-a5e9-00ffc9dcb217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:12.317452 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 59/125] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:12.317452 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e72d7a-5a57-4150-bd56-1bf093d31eb7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-1f961d25-d522-4874-a5e9-00ffc9dcb217 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:12.322882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:12.323739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:12.324479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:12.324479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:12.324479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:12.324479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:12.324479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:12.324479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:12.324479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:12.324479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:12.324479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:12.324479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:12.324479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:12.324479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:12.326716 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 67/126] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 05:59:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:12.328608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f961d25-d522-4874-a5e9-00ffc9dcb217 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:12.429378 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e72d7a-5a57-4150-bd56-1bf093d31eb7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:12.448837 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f961d25-d522-4874-a5e9-00ffc9dcb217 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:12.466293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843bced-30ad-428b-9a65-5dafa576c720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:12.467325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843bced-30ad-428b-9a65-5dafa576c720 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:12.467820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843bced-30ad-428b-9a65-5dafa576c720 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:12.468962 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843bced-30ad-428b-9a65-5dafa576c720 None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:12.470421 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843bced-30ad-428b-9a65-5dafa576c720 None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:12.472116 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843bced-30ad-428b-9a65-5dafa576c720 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:12.473960 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 60/127] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:12 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:12.474959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843bced-30ad-428b-9a65-5dafa576c720 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:12.482535 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:12.491675 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecd591-d925-483b-97ec-22875b08e905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:12.493290 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecd591-d925-483b-97ec-22875b08e905 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:12.493743 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecd591-d925-483b-97ec-22875b08e905 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:12.494207 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecd591-d925-483b-97ec-22875b08e905 None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:12.495944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecd591-d925-483b-97ec-22875b08e905 None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:12.498168 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecd591-d925-483b-97ec-22875b08e905 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:12.501939 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 68/128] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:12 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:12.502998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecd591-d925-483b-97ec-22875b08e905 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:12.557772 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:12.626635 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843bced-30ad-428b-9a65-5dafa576c720 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:12.642925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecd591-d925-483b-97ec-22875b08e905 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:12.666011 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34777a-d54d-41f2-8c8e-c492c59cc084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:12.666758 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34777a-d54d-41f2-8c8e-c492c59cc084 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:12.667722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34777a-d54d-41f2-8c8e-c492c59cc084 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:12.668070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34777a-d54d-41f2-8c8e-c492c59cc084 None admin] PATH_INFO: `/v3/users/20bcd5c59c07488b83a3391be92d86f5` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:12.669560 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34777a-d54d-41f2-8c8e-c492c59cc084 None admin] RBAC: Authorizing `identity:get_user(user_id=20bcd5c59c07488b83a3391be92d86f5)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:12.672714 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34777a-d54d-41f2-8c8e-c492c59cc084 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:12.679481 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 61/129] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:12 2022] GET /identity/v3/users/20bcd5c59c07488b83a3391be92d86f5 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:12.679481 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34777a-d54d-41f2-8c8e-c492c59cc084 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:12.689673 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29779c4c-3ab5-4d13-82ed-cd72d3d2396d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:12.690606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29779c4c-3ab5-4d13-82ed-cd72d3d2396d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:12.691129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29779c4c-3ab5-4d13-82ed-cd72d3d2396d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:12.691593 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29779c4c-3ab5-4d13-82ed-cd72d3d2396d None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:12.692446 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29779c4c-3ab5-4d13-82ed-cd72d3d2396d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:12.692959 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29779c4c-3ab5-4d13-82ed-cd72d3d2396d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:12.694611 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29779c4c-3ab5-4d13-82ed-cd72d3d2396d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:12.704019 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 69/130] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 05:59:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:12.705121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29779c4c-3ab5-4d13-82ed-cd72d3d2396d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:12.770183 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:12.814182 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34777a-d54d-41f2-8c8e-c492c59cc084 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:12.816090 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:12.819821 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29779c4c-3ab5-4d13-82ed-cd72d3d2396d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:12.864713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ea2cb-4bd6-4db1-8f42-183fc7da5086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:12.865569 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14334742-dc8f-4dab-b753-e7af00d4ce19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:12.865991 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ea2cb-4bd6-4db1-8f42-183fc7da5086 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:12.866232 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14334742-dc8f-4dab-b753-e7af00d4ce19 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:12.866569 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14334742-dc8f-4dab-b753-e7af00d4ce19 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:12.866848 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14334742-dc8f-4dab-b753-e7af00d4ce19 None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2/users/c922803daa44438cbb8185e3898e3dc9/roles/f6acb6fdca5a44de9ba00b2b0d4d3202` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:12.867090 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ea2cb-4bd6-4db1-8f42-183fc7da5086 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:12.867561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ea2cb-4bd6-4db1-8f42-183fc7da5086 None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2/users/20bcd5c59c07488b83a3391be92d86f5/roles/a2f35dad17e94553b63d07f8b8d8f1a4` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:12.869028 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14334742-dc8f-4dab-b753-e7af00d4ce19 None admin] RBAC: Authorizing `identity:create_grant(project_id=bca76726c98f48f482c676027eb393d2, user_id=c922803daa44438cbb8185e3898e3dc9, role_id=f6acb6fdca5a44de9ba00b2b0d4d3202)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:12.870148 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ea2cb-4bd6-4db1-8f42-183fc7da5086 None admin] RBAC: Authorizing `identity:create_grant(project_id=bca76726c98f48f482c676027eb393d2, user_id=20bcd5c59c07488b83a3391be92d86f5, role_id=a2f35dad17e94553b63d07f8b8d8f1a4)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:12.870494 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14334742-dc8f-4dab-b753-e7af00d4ce19 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:12.872083 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ea2cb-4bd6-4db1-8f42-183fc7da5086 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:12.893263 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 62/132] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 05:59:12 2022] PUT /identity/v3/projects/bca76726c98f48f482c676027eb393d2/users/20bcd5c59c07488b83a3391be92d86f5/roles/a2f35dad17e94553b63d07f8b8d8f1a4 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 05:59:12.894446 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ea2cb-4bd6-4db1-8f42-183fc7da5086 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:12.898616 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 70/132] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 05:59:12 2022] PUT /identity/v3/projects/bca76726c98f48f482c676027eb393d2/users/c922803daa44438cbb8185e3898e3dc9/roles/f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 05:59:12.900562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14334742-dc8f-4dab-b753-e7af00d4ce19 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:13.119206 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14334742-dc8f-4dab-b753-e7af00d4ce19 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:13.130102 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ea2cb-4bd6-4db1-8f42-183fc7da5086 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:13.165811 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:13.169467 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f95f6f-62ba-47c1-b578-f95ac81274cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:13.170009 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f95f6f-62ba-47c1-b578-f95ac81274cb None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:13.170296 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f95f6f-62ba-47c1-b578-f95ac81274cb None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:13.170669 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f95f6f-62ba-47c1-b578-f95ac81274cb None admin] PATH_INFO: `/v3/users/20bcd5c59c07488b83a3391be92d86f5` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:13.172019 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f95f6f-62ba-47c1-b578-f95ac81274cb None admin] RBAC: Authorizing `identity:get_user(user_id=20bcd5c59c07488b83a3391be92d86f5)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:13.173587 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f95f6f-62ba-47c1-b578-f95ac81274cb None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:13.175193 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd81fd3-2eb5-40be-b33b-2b9d18aae0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:13.175336 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 71/133] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:12 2022] GET /identity/v3/users/20bcd5c59c07488b83a3391be92d86f5 => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:13.176957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd81fd3-2eb5-40be-b33b-2b9d18aae0ea None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:13.176957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd81fd3-2eb5-40be-b33b-2b9d18aae0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:13.176957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd81fd3-2eb5-40be-b33b-2b9d18aae0ea None admin] PATH_INFO: `/v3/projects/5d640d7d4d914dd4b88d4203e8a0f0e0` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:13.177167 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b734415-f20a-45e8-ace2-2dba0bbd8d38 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:13.177167 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b734415-f20a-45e8-ace2-2dba0bbd8d38 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:13.177167 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b734415-f20a-45e8-ace2-2dba0bbd8d38 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:13.177570 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd81fd3-2eb5-40be-b33b-2b9d18aae0ea None admin] RBAC: Authorizing `identity:get_project(project_id=5d640d7d4d914dd4b88d4203e8a0f0e0)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:13.178358 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 72/134] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:13.179310 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd81fd3-2eb5-40be-b33b-2b9d18aae0ea None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:13.180819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 63/135] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:12 2022] GET /identity/v3/projects/5d640d7d4d914dd4b88d4203e8a0f0e0 => generated 312 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:13.185394 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3dfb1-553e-4d9a-a7d2-f25f649e508c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:13.185394 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3dfb1-553e-4d9a-a7d2-f25f649e508c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:13.185394 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3dfb1-553e-4d9a-a7d2-f25f649e508c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:13.186004 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b734415-f20a-45e8-ace2-2dba0bbd8d38 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:13.211654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-e6a3dfb1-553e-4d9a-a7d2-f25f649e508c None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:13.213305 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e6a3dfb1-553e-4d9a-a7d2-f25f649e508c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:13.227592 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:13.298878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 64/136] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:13.300025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3dfb1-553e-4d9a-a7d2-f25f649e508c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:13.330608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b734415-f20a-45e8-ace2-2dba0bbd8d38 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:13.371155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c4fd1-ec14-4385-85fe-a9f2b493a456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:13.371791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c4fd1-ec14-4385-85fe-a9f2b493a456 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:13.372072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c4fd1-ec14-4385-85fe-a9f2b493a456 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:13.372347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c4fd1-ec14-4385-85fe-a9f2b493a456 None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:13.375014 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c4fd1-ec14-4385-85fe-a9f2b493a456 None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:13.376484 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c4fd1-ec14-4385-85fe-a9f2b493a456 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:13.378346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 73/137] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:13 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:13.383833 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c4fd1-ec14-4385-85fe-a9f2b493a456 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:13.503008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3dfb1-553e-4d9a-a7d2-f25f649e508c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:13.507106 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:13.514275 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c4fd1-ec14-4385-85fe-a9f2b493a456 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:13.550490 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:13.558012 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae904ba-7e64-4549-aaa4-979488977c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:13.558716 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae904ba-7e64-4549-aaa4-979488977c4d None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:13.559131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae904ba-7e64-4549-aaa4-979488977c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:13.559428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae904ba-7e64-4549-aaa4-979488977c4d None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2/users/20bcd5c59c07488b83a3391be92d86f5/roles/f47755365f24477f9846225c9e91de2d` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:13.562600 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae904ba-7e64-4549-aaa4-979488977c4d None admin] RBAC: Authorizing `identity:create_grant(project_id=bca76726c98f48f482c676027eb393d2, user_id=20bcd5c59c07488b83a3391be92d86f5, role_id=f47755365f24477f9846225c9e91de2d)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:13.564198 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae904ba-7e64-4549-aaa4-979488977c4d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:13.567112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972caebf-fb04-464a-a52f-6fb4408d1351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:13.568097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972caebf-fb04-464a-a52f-6fb4408d1351 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:13.568677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972caebf-fb04-464a-a52f-6fb4408d1351 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:13.569133 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972caebf-fb04-464a-a52f-6fb4408d1351 None admin] PATH_INFO: `/v3/projects/5d640d7d4d914dd4b88d4203e8a0f0e0/users/20bcd5c59c07488b83a3391be92d86f5/roles/a2f35dad17e94553b63d07f8b8d8f1a4` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:13.575367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972caebf-fb04-464a-a52f-6fb4408d1351 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d640d7d4d914dd4b88d4203e8a0f0e0, user_id=20bcd5c59c07488b83a3391be92d86f5, role_id=a2f35dad17e94553b63d07f8b8d8f1a4)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:13.577175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972caebf-fb04-464a-a52f-6fb4408d1351 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:13.589377 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 74/138] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 05:59:13 2022] PUT /identity/v3/projects/bca76726c98f48f482c676027eb393d2/users/20bcd5c59c07488b83a3391be92d86f5/roles/f47755365f24477f9846225c9e91de2d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 05:59:13.604967 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 65/139] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 05:59:13 2022] PUT /identity/v3/projects/5d640d7d4d914dd4b88d4203e8a0f0e0/users/20bcd5c59c07488b83a3391be92d86f5/roles/a2f35dad17e94553b63d07f8b8d8f1a4 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 05:59:13.722334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae904ba-7e64-4549-aaa4-979488977c4d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:13.839380 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae904ba-7e64-4549-aaa4-979488977c4d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:13.879047 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb48693-0759-46b4-9fc2-1e0e07d65bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:13.880070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb48693-0759-46b4-9fc2-1e0e07d65bcf None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:13.880543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb48693-0759-46b4-9fc2-1e0e07d65bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:13.881007 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb48693-0759-46b4-9fc2-1e0e07d65bcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:13.882331 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb48693-0759-46b4-9fc2-1e0e07d65bcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:13.884076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb48693-0759-46b4-9fc2-1e0e07d65bcf None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:13.886252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 75/140] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:59:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:13.895621 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972caebf-fb04-464a-a52f-6fb4408d1351 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:14.027108 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972caebf-fb04-464a-a52f-6fb4408d1351 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:14.065313 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807ed60-f8c2-4d71-ac33-e9ab835d4f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:14.066276 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807ed60-f8c2-4d71-ac33-e9ab835d4f2d None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:14.066798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807ed60-f8c2-4d71-ac33-e9ab835d4f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:14.067268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807ed60-f8c2-4d71-ac33-e9ab835d4f2d None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:14.068136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807ed60-f8c2-4d71-ac33-e9ab835d4f2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:14.069828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807ed60-f8c2-4d71-ac33-e9ab835d4f2d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:14.096329 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 66/141] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 05:59:13 2022] POST /identity/v3/users => generated 283 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 05:59:14.308587 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd6dca-b2df-4e70-b23b-8a8ceef976d7 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:14.309344 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd6dca-b2df-4e70-b23b-8a8ceef976d7 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:14.309619 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd6dca-b2df-4e70-b23b-8a8ceef976d7 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:14.311161 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 76/142] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:14.317780 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88475f11-c637-415d-bcf6-f6319bd07ad6 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:14.318380 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88475f11-c637-415d-bcf6-f6319bd07ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:14.318895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88475f11-c637-415d-bcf6-f6319bd07ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:14.339156 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-88475f11-c637-415d-bcf6-f6319bd07ad6 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:14.340755 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-88475f11-c637-415d-bcf6-f6319bd07ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:14.393021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:14.432813 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 67/143] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:14.971347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd6dca-b2df-4e70-b23b-8a8ceef976d7 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:15.004898 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:15.105072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd6dca-b2df-4e70-b23b-8a8ceef976d7 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:15.155544 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5deda4-22e1-4de3-8b52-3dab23af7555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:15.156183 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5deda4-22e1-4de3-8b52-3dab23af7555 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:15.156514 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5deda4-22e1-4de3-8b52-3dab23af7555 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:15.156820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5deda4-22e1-4de3-8b52-3dab23af7555 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:15.167767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5deda4-22e1-4de3-8b52-3dab23af7555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-fd5deda4-22e1-4de3-8b52-3dab23af7555 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:15.174127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:15.175062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:15.175847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:15.175847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:15.175847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:15.175847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:15.175847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:15.175847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:15.175847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:15.175847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:15.175847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:15.175847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:15.175847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:15.175847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:15.175847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 77/144] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:15.183319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88475f11-c637-415d-bcf6-f6319bd07ad6 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:15.300597 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88475f11-c637-415d-bcf6-f6319bd07ad6 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:15.339513 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211fe58-eb37-4721-b940-6305bafb4cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:15.340137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211fe58-eb37-4721-b940-6305bafb4cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:15.340417 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211fe58-eb37-4721-b940-6305bafb4cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:15.340690 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211fe58-eb37-4721-b940-6305bafb4cf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:15.341369 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211fe58-eb37-4721-b940-6305bafb4cf8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:15.341674 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211fe58-eb37-4721-b940-6305bafb4cf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:15.343703 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211fe58-eb37-4721-b940-6305bafb4cf8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:15.353262 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 68/145] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:15.358746 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5deda4-22e1-4de3-8b52-3dab23af7555 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:15.469807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:15.491368 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5deda4-22e1-4de3-8b52-3dab23af7555 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:15.528210 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05e635-d591-4034-9f47-f5667b4fc867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:15.528843 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05e635-d591-4034-9f47-f5667b4fc867 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:15.529119 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05e635-d591-4034-9f47-f5667b4fc867 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:15.529388 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05e635-d591-4034-9f47-f5667b4fc867 None admin] PATH_INFO: `/v3/users/b53182f546814164abd49c1611eca319` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:15.531034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05e635-d591-4034-9f47-f5667b4fc867 None admin] RBAC: Authorizing `identity:get_user(user_id=b53182f546814164abd49c1611eca319)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:15.532535 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05e635-d591-4034-9f47-f5667b4fc867 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:15.534161 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 78/146] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:15 2022] GET /identity/v3/users/b53182f546814164abd49c1611eca319 => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:15.540063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211fe58-eb37-4721-b940-6305bafb4cf8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:15.578935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:15.653158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211fe58-eb37-4721-b940-6305bafb4cf8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:15.692248 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbf557-530a-449a-b23d-df7cadaf2d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:15.692947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbf557-530a-449a-b23d-df7cadaf2d39 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:15.693225 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbf557-530a-449a-b23d-df7cadaf2d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:15.693496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbf557-530a-449a-b23d-df7cadaf2d39 None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:15.694643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbf557-530a-449a-b23d-df7cadaf2d39 None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:15.696099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbf557-530a-449a-b23d-df7cadaf2d39 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:15.698128 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 69/147] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:15 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:15.703793 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05e635-d591-4034-9f47-f5667b4fc867 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:15.822304 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05e635-d591-4034-9f47-f5667b4fc867 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:15.856702 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8906f9-353e-4d2a-81a1-a95fa12eed3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:15.857290 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8906f9-353e-4d2a-81a1-a95fa12eed3c None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:15.857569 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8906f9-353e-4d2a-81a1-a95fa12eed3c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:15.862953 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8906f9-353e-4d2a-81a1-a95fa12eed3c None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2/users/b53182f546814164abd49c1611eca319/roles/86e796f30478458d99f1d139c853a1db` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:15.864963 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8906f9-353e-4d2a-81a1-a95fa12eed3c None admin] RBAC: Authorizing `identity:create_grant(project_id=bca76726c98f48f482c676027eb393d2, user_id=b53182f546814164abd49c1611eca319, role_id=86e796f30478458d99f1d139c853a1db)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:15.866436 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8906f9-353e-4d2a-81a1-a95fa12eed3c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:15.880917 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 79/148] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 05:59:15 2022] PUT /identity/v3/projects/bca76726c98f48f482c676027eb393d2/users/b53182f546814164abd49c1611eca319/roles/86e796f30478458d99f1d139c853a1db => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 05:59:15.885110 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e2388-38e6-4cb3-bbe2-4f66c672a821 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:15.885402 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e2388-38e6-4cb3-bbe2-4f66c672a821 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:15.885657 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e2388-38e6-4cb3-bbe2-4f66c672a821 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:15.887243 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 70/149] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:15.894235 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e443f-5c7f-48f9-8609-72f492af9a9e None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:15.894891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e443f-5c7f-48f9-8609-72f492af9a9e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:15.895367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e443f-5c7f-48f9-8609-72f492af9a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:15.917780 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b61e443f-5c7f-48f9-8609-72f492af9a9e None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:15.919393 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b61e443f-5c7f-48f9-8609-72f492af9a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:15.928379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:16.008469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99108fe1-4bbe-4d33-981c-4e6a4bd08aaa None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:16.009186 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99108fe1-4bbe-4d33-981c-4e6a4bd08aaa None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:16.009778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99108fe1-4bbe-4d33-981c-4e6a4bd08aaa None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:16.012230 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 71/150] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:16.017867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb99f4e-e4cf-43f9-bd48-21a6825c5d66 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:16.018425 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb99f4e-e4cf-43f9-bd48-21a6825c5d66 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:16.019323 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb99f4e-e4cf-43f9-bd48-21a6825c5d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:16.029377 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 80/151] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:16.049674 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-5eb99f4e-e4cf-43f9-bd48-21a6825c5d66 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:16.051307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5eb99f4e-e4cf-43f9-bd48-21a6825c5d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:16.130484 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 72/152] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:16.617592 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e443f-5c7f-48f9-8609-72f492af9a9e None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:16.708955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb99f4e-e4cf-43f9-bd48-21a6825c5d66 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:16.767350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:16.775376 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e443f-5c7f-48f9-8609-72f492af9a9e None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:16.842823 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496397bf-ff8f-4962-93f7-1bf48a8a37c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:16.844396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496397bf-ff8f-4962-93f7-1bf48a8a37c4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:16.845479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496397bf-ff8f-4962-93f7-1bf48a8a37c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:16.846065 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496397bf-ff8f-4962-93f7-1bf48a8a37c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:16.881955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496397bf-ff8f-4962-93f7-1bf48a8a37c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-496397bf-ff8f-4962-93f7-1bf48a8a37c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:16.888559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:16.889576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:16.890692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:16.890692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:16.890692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:16.890692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:16.890692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:16.890692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:16.890692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:16.890692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:16.890692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:16.890692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:16.890692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:16.890692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:16.893930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 81/153] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:16.897879 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2a231-f422-466c-ae8f-d2a9267c3993 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:16.900573 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2a231-f422-466c-ae8f-d2a9267c3993 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:16.901758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2a231-f422-466c-ae8f-d2a9267c3993 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:16.903949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 82/154] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:16.908064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb99f4e-e4cf-43f9-bd48-21a6825c5d66 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:16.911003 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc468622-77bb-40f2-b7ba-6dd0017501b7 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:16.911628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc468622-77bb-40f2-b7ba-6dd0017501b7 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:16.912697 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc468622-77bb-40f2-b7ba-6dd0017501b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:16.929597 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:16.943754 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-bc468622-77bb-40f2-b7ba-6dd0017501b7 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:16.945306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bc468622-77bb-40f2-b7ba-6dd0017501b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:17.004139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa9df0-6102-490a-83fc-517e0f886653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:17.004795 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa9df0-6102-490a-83fc-517e0f886653 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:17.005106 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa9df0-6102-490a-83fc-517e0f886653 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:17.005404 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa9df0-6102-490a-83fc-517e0f886653 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:17.025105 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa9df0-6102-490a-83fc-517e0f886653 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-a9aa9df0-6102-490a-83fc-517e0f886653 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:17.030086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:17.031689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:17.032577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:17.032577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:17.032577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:17.032577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:17.032577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:17.032577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:17.032577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:17.032577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:17.032577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:17.032577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:17.032577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:17.032577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:17.032577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 73/155] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 05:59:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:17.033268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa9df0-6102-490a-83fc-517e0f886653 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:17.082061 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 83/156] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:17.084023 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fb8d2-2bad-4651-9cc4-83ade80d3b2f None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:17.084507 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fb8d2-2bad-4651-9cc4-83ade80d3b2f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:17.084977 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fb8d2-2bad-4651-9cc4-83ade80d3b2f None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:17.086428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 84/157] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:17.087328 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fb8d2-2bad-4651-9cc4-83ade80d3b2f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:17.174968 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:17.188804 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa9df0-6102-490a-83fc-517e0f886653 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:17.248611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d172c58-9d90-4bc6-bc0c-10cd267bea45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:17.249323 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d172c58-9d90-4bc6-bc0c-10cd267bea45 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:17.249639 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d172c58-9d90-4bc6-bc0c-10cd267bea45 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:17.249941 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d172c58-9d90-4bc6-bc0c-10cd267bea45 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:17.250789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d172c58-9d90-4bc6-bc0c-10cd267bea45 None admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:17.251098 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d172c58-9d90-4bc6-bc0c-10cd267bea45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:17.252508 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d172c58-9d90-4bc6-bc0c-10cd267bea45 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:17.266273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 74/158] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:17.268221 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c6c10-e5c9-4ace-b85f-f225d09f6f73 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:17.268221 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c6c10-e5c9-4ace-b85f-f225d09f6f73 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:17.268221 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c6c10-e5c9-4ace-b85f-f225d09f6f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:17.307625 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-470c6c10-e5c9-4ace-b85f-f225d09f6f73 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:17.308865 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-470c6c10-e5c9-4ace-b85f-f225d09f6f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:17.346467 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fb8d2-2bad-4651-9cc4-83ade80d3b2f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:17.412395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:17.417627 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ff048-f8f5-4554-9df3-3ccdd2560699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:17.418244 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ff048-f8f5-4554-9df3-3ccdd2560699 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:17.418535 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ff048-f8f5-4554-9df3-3ccdd2560699 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:17.418818 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ff048-f8f5-4554-9df3-3ccdd2560699 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:17.419490 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ff048-f8f5-4554-9df3-3ccdd2560699 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:17.420076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ff048-f8f5-4554-9df3-3ccdd2560699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:17.420176 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 75/159] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:17.420693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c6c10-e5c9-4ace-b85f-f225d09f6f73 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:17.421812 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ff048-f8f5-4554-9df3-3ccdd2560699 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:17.453405 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 85/160] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:17.458632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ff048-f8f5-4554-9df3-3ccdd2560699 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:17.562186 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:17.569338 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c6c10-e5c9-4ace-b85f-f225d09f6f73 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:17.627804 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53a063-aeac-4d4e-87f3-edc03066d413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:17.628450 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53a063-aeac-4d4e-87f3-edc03066d413 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:17.628730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53a063-aeac-4d4e-87f3-edc03066d413 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:17.629006 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53a063-aeac-4d4e-87f3-edc03066d413 None admin] PATH_INFO: `/v3/users/20bcd5c59c07488b83a3391be92d86f5` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:17.635237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53a063-aeac-4d4e-87f3-edc03066d413 None admin] RBAC: Authorizing `identity:get_user(user_id=20bcd5c59c07488b83a3391be92d86f5)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:17.636990 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53a063-aeac-4d4e-87f3-edc03066d413 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:17.642585 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 76/161] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:17 2022] GET /identity/v3/users/20bcd5c59c07488b83a3391be92d86f5 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:17.661690 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53a063-aeac-4d4e-87f3-edc03066d413 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:17.664834 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ff048-f8f5-4554-9df3-3ccdd2560699 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:17.721195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379eafd-9f14-4743-bf19-a0c3ce10277a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:17.722199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379eafd-9f14-4743-bf19-a0c3ce10277a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:17.722708 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379eafd-9f14-4743-bf19-a0c3ce10277a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:17.723184 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379eafd-9f14-4743-bf19-a0c3ce10277a None admin] PATH_INFO: `/v3/users/c922803daa44438cbb8185e3898e3dc9` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:17.725200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379eafd-9f14-4743-bf19-a0c3ce10277a None admin] RBAC: Authorizing `identity:get_user(user_id=c922803daa44438cbb8185e3898e3dc9)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:17.726948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379eafd-9f14-4743-bf19-a0c3ce10277a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:17.728773 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 86/162] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:17 2022] GET /identity/v3/users/c922803daa44438cbb8185e3898e3dc9 => generated 247 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:17.729736 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379eafd-9f14-4743-bf19-a0c3ce10277a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:17.852571 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:17.869424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53a063-aeac-4d4e-87f3-edc03066d413 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:17.932734 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379eafd-9f14-4743-bf19-a0c3ce10277a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:17.954691 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f93b1-2bb5-40db-aa30-40012c9766a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:17.955306 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f93b1-2bb5-40db-aa30-40012c9766a7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:17.955627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f93b1-2bb5-40db-aa30-40012c9766a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:17.955847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f93b1-2bb5-40db-aa30-40012c9766a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:17.997703 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702cc334-4c31-49bd-a036-431137f6bce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:17.998261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702cc334-4c31-49bd-a036-431137f6bce7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:17.998576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702cc334-4c31-49bd-a036-431137f6bce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:17.998852 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702cc334-4c31-49bd-a036-431137f6bce7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:18.008683 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:18.024049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702cc334-4c31-49bd-a036-431137f6bce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-702cc334-4c31-49bd-a036-431137f6bce7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:18.029510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:18.030589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:18.031638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:18.031638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:18.031638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:18.031638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:18.031638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:18.031638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:18.031638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:18.031638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:18.031638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:18.031638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:18.031638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:18.031638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:18.033502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 87/163] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 05:59:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:18.037998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702cc334-4c31-49bd-a036-431137f6bce7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:18.124952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f93b1-2bb5-40db-aa30-40012c9766a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:18.127023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f93b1-2bb5-40db-aa30-40012c9766a7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:18.285597 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702cc334-4c31-49bd-a036-431137f6bce7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:18.287518 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 77/164] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 05:59:18.295777 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f93b1-2bb5-40db-aa30-40012c9766a7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:18.321794 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:18.352185 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457af5ba-0320-4296-bbb1-a10b068db5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:18.352779 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457af5ba-0320-4296-bbb1-a10b068db5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:18.353062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457af5ba-0320-4296-bbb1-a10b068db5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:18.353332 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457af5ba-0320-4296-bbb1-a10b068db5f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:18.360307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:18.416723 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f93b1-2bb5-40db-aa30-40012c9766a7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:18.466375 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c479594-edf8-4981-8268-e3146536b154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:18.467469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c479594-edf8-4981-8268-e3146536b154 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:18.468080 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c479594-edf8-4981-8268-e3146536b154 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:18.468992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c479594-edf8-4981-8268-e3146536b154 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:18.470101 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c479594-edf8-4981-8268-e3146536b154 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:18.470687 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c479594-edf8-4981-8268-e3146536b154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:18.472280 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c479594-edf8-4981-8268-e3146536b154 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:18.481510 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 78/165] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 05:59:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:18.482864 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c479594-edf8-4981-8268-e3146536b154 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:18.492537 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457af5ba-0320-4296-bbb1-a10b068db5f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:18.494394 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457af5ba-0320-4296-bbb1-a10b068db5f9 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:18.609946 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c479594-edf8-4981-8268-e3146536b154 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:18.635053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:18.650636 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 88/166] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:18.651262 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457af5ba-0320-4296-bbb1-a10b068db5f9 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:18.668668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2b106-d3dc-4a4c-bc83-1a9201708c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:18.669678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2b106-d3dc-4a4c-bc83-1a9201708c13 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:18.670246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2b106-d3dc-4a4c-bc83-1a9201708c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:18.670764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2b106-d3dc-4a4c-bc83-1a9201708c13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:18.686352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:18.694033 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2b106-d3dc-4a4c-bc83-1a9201708c13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-bba2b106-d3dc-4a4c-bc83-1a9201708c13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:18.700049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:18.700931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:18.701705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:18.701705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:18.701705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:18.701705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:18.701705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:18.701705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:18.701705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:18.701705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:18.701705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:18.701705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:18.701705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:18.701705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:18.705473 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 79/167] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:18.706548 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2b106-d3dc-4a4c-bc83-1a9201708c13 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:18.780030 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457af5ba-0320-4296-bbb1-a10b068db5f9 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:18.832294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4ccac-bcdb-4fad-ac0f-22a7edc81333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:18.833013 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4ccac-bcdb-4fad-ac0f-22a7edc81333 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:18.833297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4ccac-bcdb-4fad-ac0f-22a7edc81333 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:18.833574 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4ccac-bcdb-4fad-ac0f-22a7edc81333 None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:18.839305 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4ccac-bcdb-4fad-ac0f-22a7edc81333 None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:18.840802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4ccac-bcdb-4fad-ac0f-22a7edc81333 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:18.842548 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 89/168] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:18 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:18.843486 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4ccac-bcdb-4fad-ac0f-22a7edc81333 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:18.880942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2b106-d3dc-4a4c-bc83-1a9201708c13 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:18.928715 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfcd28-22de-4150-8ec2-eed2b7de8816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:18.929885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfcd28-22de-4150-8ec2-eed2b7de8816 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:18.930453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfcd28-22de-4150-8ec2-eed2b7de8816 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:18.930987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfcd28-22de-4150-8ec2-eed2b7de8816 None admin] PATH_INFO: `/v3/users/20bcd5c59c07488b83a3391be92d86f5` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:18.935214 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfcd28-22de-4150-8ec2-eed2b7de8816 None admin] RBAC: Authorizing `identity:get_user(user_id=20bcd5c59c07488b83a3391be92d86f5)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:18.937868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfcd28-22de-4150-8ec2-eed2b7de8816 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:18.940138 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 80/169] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:18 2022] GET /identity/v3/users/20bcd5c59c07488b83a3391be92d86f5 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:18.941132 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfcd28-22de-4150-8ec2-eed2b7de8816 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:18.981905 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:18.998403 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4ccac-bcdb-4fad-ac0f-22a7edc81333 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:19.003367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:19.067569 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfcd28-22de-4150-8ec2-eed2b7de8816 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:19.090591 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4bf710-dfb2-435b-9579-d44650d4d23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:19.091251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bf710-dfb2-435b-9579-d44650d4d23c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:19.091566 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bf710-dfb2-435b-9579-d44650d4d23c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:19.091767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bf710-dfb2-435b-9579-d44650d4d23c None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:19.093426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4bf710-dfb2-435b-9579-d44650d4d23c None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:19.108219 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4bf710-dfb2-435b-9579-d44650d4d23c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:19.108219 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 90/170] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:18 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:19.108219 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4bf710-dfb2-435b-9579-d44650d4d23c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:19.116180 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2a742-d08b-4528-a4ba-b04c726621e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:19.117163 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2a742-d08b-4528-a4ba-b04c726621e4 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:19.117894 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2a742-d08b-4528-a4ba-b04c726621e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:19.118375 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2a742-d08b-4528-a4ba-b04c726621e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:19.119279 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2a742-d08b-4528-a4ba-b04c726621e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:19.119871 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2a742-d08b-4528-a4ba-b04c726621e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:19.121470 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2a742-d08b-4528-a4ba-b04c726621e4 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:19.134333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 81/171] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:19.135633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2a742-d08b-4528-a4ba-b04c726621e4 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:19.263541 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2a742-d08b-4528-a4ba-b04c726621e4 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:19.274605 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:19.275834 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4bf710-dfb2-435b-9579-d44650d4d23c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:19.308345 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b99ed6-2300-4e0f-ae34-11ecbff54e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:19.308957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b99ed6-2300-4e0f-ae34-11ecbff54e57 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:19.309237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b99ed6-2300-4e0f-ae34-11ecbff54e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:19.309691 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b99ed6-2300-4e0f-ae34-11ecbff54e57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:19.314944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf6fa2-a9a8-4325-800b-9c32104fb963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:19.315561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf6fa2-a9a8-4325-800b-9c32104fb963 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:19.315824 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf6fa2-a9a8-4325-800b-9c32104fb963 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:19.316093 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf6fa2-a9a8-4325-800b-9c32104fb963 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:19.316827 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf6fa2-a9a8-4325-800b-9c32104fb963 None admin] RBAC: Adding query filter params (role.id=a2f35dad17e94553b63d07f8b8d8f1a4, scope.project.id=bca76726c98f48f482c676027eb393d2, user.id=20bcd5c59c07488b83a3391be92d86f5) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:19.317132 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf6fa2-a9a8-4325-800b-9c32104fb963 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:19.318606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf6fa2-a9a8-4325-800b-9c32104fb963 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:19.325025 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 91/172] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:19 2022] GET /identity/v3/role_assignments?user.id=20bcd5c59c07488b83a3391be92d86f5&scope.project.id=bca76726c98f48f482c676027eb393d2&role.id=a2f35dad17e94553b63d07f8b8d8f1a4 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:19.326090 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbeb563-b514-442c-ab09-dd08ab0f227c None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:19.326372 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbeb563-b514-442c-ab09-dd08ab0f227c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:19.326677 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbeb563-b514-442c-ab09-dd08ab0f227c None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:19.328607 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 92/173] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:19.329185 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbeb563-b514-442c-ab09-dd08ab0f227c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:19.366059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:19.407292 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b99ed6-2300-4e0f-ae34-11ecbff54e57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:19.408710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b99ed6-2300-4e0f-ae34-11ecbff54e57 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:19.482968 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbeb563-b514-442c-ab09-dd08ab0f227c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:19.531414 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ef66d-9894-4398-b04d-25ba3ce4a3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:19.532033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ef66d-9894-4398-b04d-25ba3ce4a3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:19.532311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ef66d-9894-4398-b04d-25ba3ce4a3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:19.532584 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ef66d-9894-4398-b04d-25ba3ce4a3c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:19.533325 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ef66d-9894-4398-b04d-25ba3ce4a3c9 None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=bca76726c98f48f482c676027eb393d2, user.id=c922803daa44438cbb8185e3898e3dc9) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:19.533637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ef66d-9894-4398-b04d-25ba3ce4a3c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:19.535129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ef66d-9894-4398-b04d-25ba3ce4a3c9 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:19.541268 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 93/174] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:19 2022] GET /identity/v3/role_assignments?user.id=c922803daa44438cbb8185e3898e3dc9&scope.project.id=bca76726c98f48f482c676027eb393d2&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:19.542181 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ef66d-9894-4398-b04d-25ba3ce4a3c9 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:19.555151 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 82/175] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:19.556910 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b3365-09f6-4715-80d9-9eb393f78abb None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:19.557247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b3365-09f6-4715-80d9-9eb393f78abb None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:19.557512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b3365-09f6-4715-80d9-9eb393f78abb None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:19.560255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 83/176] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:19.562656 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c55a34-58a1-4526-be18-600dbc934cf2 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:19.563151 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c55a34-58a1-4526-be18-600dbc934cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:19.563623 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c55a34-58a1-4526-be18-600dbc934cf2 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:19.570166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 84/177] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:19.571522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee5876-7592-4ca6-af8a-4a285037e5fa None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:19.571867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee5876-7592-4ca6-af8a-4a285037e5fa None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:19.572260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee5876-7592-4ca6-af8a-4a285037e5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:19.586621 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:19.593629 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-dcee5876-7592-4ca6-af8a-4a285037e5fa None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:19.595733 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-dcee5876-7592-4ca6-af8a-4a285037e5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:19.650400 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:19.670267 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ef66d-9894-4398-b04d-25ba3ce4a3c9 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:19.678052 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 85/178] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:19.679204 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3f8e-42da-4aa0-bef8-1111b9d9a9d9 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:19.679484 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3f8e-42da-4aa0-bef8-1111b9d9a9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:19.679752 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3f8e-42da-4aa0-bef8-1111b9d9a9d9 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:19.685188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 86/179] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:19.685188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab3f8e-42da-4aa0-bef8-1111b9d9a9d9 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:19.722974 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6b635-1d0d-47f9-88d1-513e87de1d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:19.723175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6b635-1d0d-47f9-88d1-513e87de1d75 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:19.723554 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6b635-1d0d-47f9-88d1-513e87de1d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:19.723827 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6b635-1d0d-47f9-88d1-513e87de1d75 None admin] PATH_INFO: `/v3/users/20bcd5c59c07488b83a3391be92d86f5` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:19.724962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6b635-1d0d-47f9-88d1-513e87de1d75 None admin] RBAC: Authorizing `identity:get_user(user_id=20bcd5c59c07488b83a3391be92d86f5)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:19.727227 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6b635-1d0d-47f9-88d1-513e87de1d75 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:19.729251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 94/180] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:19 2022] GET /identity/v3/users/20bcd5c59c07488b83a3391be92d86f5 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:19.730000 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3430f88-d4ed-45fe-aad2-a3492ff46855 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:19.730275 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3430f88-d4ed-45fe-aad2-a3492ff46855 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:19.734673 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3430f88-d4ed-45fe-aad2-a3492ff46855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:19.756895 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-f3430f88-d4ed-45fe-aad2-a3492ff46855 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:19.758986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f3430f88-d4ed-45fe-aad2-a3492ff46855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:19.822717 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab3f8e-42da-4aa0-bef8-1111b9d9a9d9 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:19.845816 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 95/181] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:19.855310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81c91b-de50-4d83-a3a5-5c30622cadfa None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:19.855669 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81c91b-de50-4d83-a3a5-5c30622cadfa None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:19.856363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81c91b-de50-4d83-a3a5-5c30622cadfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:19.865146 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0e5cb-6d36-4838-a25c-7bf6e09a766d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:19.865802 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0e5cb-6d36-4838-a25c-7bf6e09a766d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:19.866786 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0e5cb-6d36-4838-a25c-7bf6e09a766d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:19.867078 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0e5cb-6d36-4838-a25c-7bf6e09a766d None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:19.871381 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0e5cb-6d36-4838-a25c-7bf6e09a766d None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:19.873302 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0e5cb-6d36-4838-a25c-7bf6e09a766d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:19.875917 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 87/182] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:19 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:19.877093 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bc403-4796-4fe9-abe1-7a92e7542f78 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:19.879393 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-6d81c91b-de50-4d83-a3a5-5c30622cadfa None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:19.880586 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6d81c91b-de50-4d83-a3a5-5c30622cadfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:19.882125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bc403-4796-4fe9-abe1-7a92e7542f78 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:19.882594 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bc403-4796-4fe9-abe1-7a92e7542f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:19.894126 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:19.910640 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-251bc403-4796-4fe9-abe1-7a92e7542f78 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:19.912345 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-251bc403-4796-4fe9-abe1-7a92e7542f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:19.975525 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 96/183] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:19.982608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81c91b-de50-4d83-a3a5-5c30622cadfa None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:19.996981 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:20.010660 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 88/184] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:20.011722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bc403-4796-4fe9-abe1-7a92e7542f78 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:20.158379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81c91b-de50-4d83-a3a5-5c30622cadfa None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:20.161990 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bc403-4796-4fe9-abe1-7a92e7542f78 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:20.197033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df60f2-51ee-452b-80e9-6f7bf6811f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:20.197631 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df60f2-51ee-452b-80e9-6f7bf6811f36 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:20.197909 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df60f2-51ee-452b-80e9-6f7bf6811f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:20.198181 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df60f2-51ee-452b-80e9-6f7bf6811f36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:20.208453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d91a17-5d42-40f0-b004-4e97d3b50053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:20.209350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d91a17-5d42-40f0-b004-4e97d3b50053 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:20.210294 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d91a17-5d42-40f0-b004-4e97d3b50053 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:20.210791 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d91a17-5d42-40f0-b004-4e97d3b50053 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:20.212290 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d91a17-5d42-40f0-b004-4e97d3b50053 None admin] RBAC: Adding query filter params (role.id=f47755365f24477f9846225c9e91de2d, scope.project.id=bca76726c98f48f482c676027eb393d2, user.id=20bcd5c59c07488b83a3391be92d86f5) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:20.212876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d91a17-5d42-40f0-b004-4e97d3b50053 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:20.214824 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d91a17-5d42-40f0-b004-4e97d3b50053 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:20.217627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:20.224049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 89/185] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:20 2022] GET /identity/v3/role_assignments?user.id=20bcd5c59c07488b83a3391be92d86f5&scope.project.id=bca76726c98f48f482c676027eb393d2&role.id=f47755365f24477f9846225c9e91de2d => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:20.225370 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d91a17-5d42-40f0-b004-4e97d3b50053 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:20.285145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df60f2-51ee-452b-80e9-6f7bf6811f36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:20.287538 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df60f2-51ee-452b-80e9-6f7bf6811f36 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:20.313543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:20.357280 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d91a17-5d42-40f0-b004-4e97d3b50053 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:20.399889 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9128b54-b5ea-4373-bcc4-c39b73c424ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:20.400761 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9128b54-b5ea-4373-bcc4-c39b73c424ff None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:20.401210 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9128b54-b5ea-4373-bcc4-c39b73c424ff None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:20.401479 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9128b54-b5ea-4373-bcc4-c39b73c424ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:20.412742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9128b54-b5ea-4373-bcc4-c39b73c424ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-b9128b54-b5ea-4373-bcc4-c39b73c424ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:20.417947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:20.418856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:20.419636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:20.419636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:20.419636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:20.419636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:20.419636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:20.419636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:20.419636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:20.419636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:20.419636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:20.419636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:20.419636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:20.419636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:20.419636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 90/186] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 05:59:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:20.420298 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9128b54-b5ea-4373-bcc4-c39b73c424ff None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:20.432121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 97/187] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:20.433363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df60f2-51ee-452b-80e9-6f7bf6811f36 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:20.500185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:20.547538 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df60f2-51ee-452b-80e9-6f7bf6811f36 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:20.548161 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9128b54-b5ea-4373-bcc4-c39b73c424ff None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:20.562193 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:20.583584 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ef90e-49bd-4a36-a7fa-c5871f900bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:20.584143 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ef90e-49bd-4a36-a7fa-c5871f900bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:20.584465 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ef90e-49bd-4a36-a7fa-c5871f900bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:20.584560 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6da2b-c7a0-4ab0-b245-c685635fbc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:20.584794 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ef90e-49bd-4a36-a7fa-c5871f900bb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:20.585080 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6da2b-c7a0-4ab0-b245-c685635fbc8f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:20.585361 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6da2b-c7a0-4ab0-b245-c685635fbc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:20.585631 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6da2b-c7a0-4ab0-b245-c685635fbc8f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:20.585789 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ef90e-49bd-4a36-a7fa-c5871f900bb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:20.587233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ef90e-49bd-4a36-a7fa-c5871f900bb4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:20.588789 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 98/188] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:59:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:20.590789 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ef90e-49bd-4a36-a7fa-c5871f900bb4 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:20.600415 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6da2b-c7a0-4ab0-b245-c685635fbc8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-79b6da2b-c7a0-4ab0-b245-c685635fbc8f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:20.605266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:20.606278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:20.607506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:20.607506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:20.607506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:20.607506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:20.607506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:20.607506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:20.607506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:20.607506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:20.607506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:20.607506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:20.607506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:20.607506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:20.607506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 91/189] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 05:59:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:20.609965 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6da2b-c7a0-4ab0-b245-c685635fbc8f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:20.689210 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ef90e-49bd-4a36-a7fa-c5871f900bb4 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:20.710217 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6da2b-c7a0-4ab0-b245-c685635fbc8f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:20.723395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b88ea3-932e-46c9-a264-1cfd4ebe5df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:20.724040 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b88ea3-932e-46c9-a264-1cfd4ebe5df9 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:20.724315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b88ea3-932e-46c9-a264-1cfd4ebe5df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:20.724580 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b88ea3-932e-46c9-a264-1cfd4ebe5df9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:20.727637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:20.735666 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b88ea3-932e-46c9-a264-1cfd4ebe5df9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-d2b88ea3-932e-46c9-a264-1cfd4ebe5df9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:20.740719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:20.741758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:20.742856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:20.742856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:20.742856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:20.742856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:20.742856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:20.742856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:20.742856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:20.742856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:20.742856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:20.742856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:20.742856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:20.742856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:20.742856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 99/190] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:20.743504 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b88ea3-932e-46c9-a264-1cfd4ebe5df9 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:20.749453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3e8b5-4533-4eea-9819-5cd4f74d8998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:20.750278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3e8b5-4533-4eea-9819-5cd4f74d8998 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:20.750498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3e8b5-4533-4eea-9819-5cd4f74d8998 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:20.750791 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3e8b5-4533-4eea-9819-5cd4f74d8998 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:20.751714 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3e8b5-4533-4eea-9819-5cd4f74d8998 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:20.752180 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3e8b5-4533-4eea-9819-5cd4f74d8998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:20.753945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3e8b5-4533-4eea-9819-5cd4f74d8998 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:20.762768 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 92/191] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:20.763546 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3e8b5-4533-4eea-9819-5cd4f74d8998 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:20.788407 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:20.850340 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b88ea3-932e-46c9-a264-1cfd4ebe5df9 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:20.876180 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3e8b5-4533-4eea-9819-5cd4f74d8998 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:20.885120 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b0063-5ac5-40da-bebc-e8a528f101b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:20.885745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b0063-5ac5-40da-bebc-e8a528f101b3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:20.886021 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b0063-5ac5-40da-bebc-e8a528f101b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:20.886294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b0063-5ac5-40da-bebc-e8a528f101b3 None admin] PATH_INFO: `/v3/projects/5d640d7d4d914dd4b88d4203e8a0f0e0` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:20.887436 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b0063-5ac5-40da-bebc-e8a528f101b3 None admin] RBAC: Authorizing `identity:get_project(project_id=5d640d7d4d914dd4b88d4203e8a0f0e0)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:20.888880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b0063-5ac5-40da-bebc-e8a528f101b3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:20.891179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 100/192] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:20 2022] GET /identity/v3/projects/5d640d7d4d914dd4b88d4203e8a0f0e0 => generated 312 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:20.891858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b0063-5ac5-40da-bebc-e8a528f101b3 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:20.916375 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aff2e4-2a2f-4a81-8980-92858b960815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:20.916982 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aff2e4-2a2f-4a81-8980-92858b960815 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:20.917260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aff2e4-2a2f-4a81-8980-92858b960815 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:20.917531 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aff2e4-2a2f-4a81-8980-92858b960815 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:20.918175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aff2e4-2a2f-4a81-8980-92858b960815 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:20.919796 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aff2e4-2a2f-4a81-8980-92858b960815 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:20.940621 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 93/193] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 05:59:20 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 05:59:20.941454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aff2e4-2a2f-4a81-8980-92858b960815 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:21.003762 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:21.013547 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b0063-5ac5-40da-bebc-e8a528f101b3 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:21.027102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:21.053071 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8153b89-8583-4fc8-8fe8-edc51e610879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:21.053683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8153b89-8583-4fc8-8fe8-edc51e610879 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:21.053956 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8153b89-8583-4fc8-8fe8-edc51e610879 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:21.054221 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8153b89-8583-4fc8-8fe8-edc51e610879 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:21.054942 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8153b89-8583-4fc8-8fe8-edc51e610879 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:21.055246 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8153b89-8583-4fc8-8fe8-edc51e610879 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:21.056657 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8153b89-8583-4fc8-8fe8-edc51e610879 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:21.070318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 101/194] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 05:59:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:21.070318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8153b89-8583-4fc8-8fe8-edc51e610879 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:21.089968 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aff2e4-2a2f-4a81-8980-92858b960815 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:21.139150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e6b16-44bb-4633-8aa3-27da89b8fc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:21.141495 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e6b16-44bb-4633-8aa3-27da89b8fc5f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:21.142184 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e6b16-44bb-4633-8aa3-27da89b8fc5f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:21.142408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e6b16-44bb-4633-8aa3-27da89b8fc5f None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:21.143118 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e6b16-44bb-4633-8aa3-27da89b8fc5f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:21.143427 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e6b16-44bb-4633-8aa3-27da89b8fc5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:21.146548 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e6b16-44bb-4633-8aa3-27da89b8fc5f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:21.155977 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 94/195] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:21.156989 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e6b16-44bb-4633-8aa3-27da89b8fc5f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:21.206183 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8153b89-8583-4fc8-8fe8-edc51e610879 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:21.256489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fbede-410c-4ca3-9baf-92c8b4320a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:21.257048 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fbede-410c-4ca3-9baf-92c8b4320a58 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:21.257325 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fbede-410c-4ca3-9baf-92c8b4320a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:21.257594 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fbede-410c-4ca3-9baf-92c8b4320a58 None admin] PATH_INFO: `/v3/users/bfb2ee6a3afe43c78883d3930589562c` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:21.273199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fbede-410c-4ca3-9baf-92c8b4320a58 None admin] RBAC: Authorizing `identity:get_user(user_id=bfb2ee6a3afe43c78883d3930589562c)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:21.275699 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fbede-410c-4ca3-9baf-92c8b4320a58 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:21.277404 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 102/196] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:21 2022] GET /identity/v3/users/bfb2ee6a3afe43c78883d3930589562c => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:21.278715 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fbede-410c-4ca3-9baf-92c8b4320a58 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:21.288806 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:21.291903 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e6b16-44bb-4633-8aa3-27da89b8fc5f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:21.319480 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:21.336909 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b01cb5-798d-473d-99d5-c5be57f82b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:21.338623 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b01cb5-798d-473d-99d5-c5be57f82b4d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:21.339044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b01cb5-798d-473d-99d5-c5be57f82b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:21.339446 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b01cb5-798d-473d-99d5-c5be57f82b4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:21.340711 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b01cb5-798d-473d-99d5-c5be57f82b4d None admin] RBAC: Adding query filter params (role.id=a2f35dad17e94553b63d07f8b8d8f1a4, scope.project.id=5d640d7d4d914dd4b88d4203e8a0f0e0, user.id=20bcd5c59c07488b83a3391be92d86f5) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:21.341344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b01cb5-798d-473d-99d5-c5be57f82b4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:21.343905 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b01cb5-798d-473d-99d5-c5be57f82b4d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:21.353952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 95/197] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:21 2022] GET /identity/v3/role_assignments?user.id=20bcd5c59c07488b83a3391be92d86f5&scope.project.id=5d640d7d4d914dd4b88d4203e8a0f0e0&role.id=a2f35dad17e94553b63d07f8b8d8f1a4 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:21.353952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b01cb5-798d-473d-99d5-c5be57f82b4d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:21.423241 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fbede-410c-4ca3-9baf-92c8b4320a58 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:21.473926 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04883258-7691-44a9-a8ac-e0ee7de4bf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:21.474863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04883258-7691-44a9-a8ac-e0ee7de4bf27 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:21.475138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04883258-7691-44a9-a8ac-e0ee7de4bf27 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:21.475406 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04883258-7691-44a9-a8ac-e0ee7de4bf27 None admin] PATH_INFO: `/v3/users/bfb2ee6a3afe43c78883d3930589562c` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:21.476232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b01cb5-798d-473d-99d5-c5be57f82b4d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:21.476555 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04883258-7691-44a9-a8ac-e0ee7de4bf27 None admin] RBAC: Authorizing `identity:get_user(user_id=bfb2ee6a3afe43c78883d3930589562c)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:21.478290 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04883258-7691-44a9-a8ac-e0ee7de4bf27 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:21.481215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 103/198] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:21 2022] GET /identity/v3/users/bfb2ee6a3afe43c78883d3930589562c => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:21.482010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04883258-7691-44a9-a8ac-e0ee7de4bf27 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:21.526630 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab81933-10eb-41b3-989a-6efee743f030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:21.528116 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab81933-10eb-41b3-989a-6efee743f030 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:21.528399 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab81933-10eb-41b3-989a-6efee743f030 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:21.528671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab81933-10eb-41b3-989a-6efee743f030 None admin] PATH_INFO: `/v3/users/b53182f546814164abd49c1611eca319` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:21.529838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab81933-10eb-41b3-989a-6efee743f030 None admin] RBAC: Authorizing `identity:get_user(user_id=b53182f546814164abd49c1611eca319)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:21.531845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab81933-10eb-41b3-989a-6efee743f030 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:21.533516 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 96/199] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:21 2022] GET /identity/v3/users/b53182f546814164abd49c1611eca319 => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:21.535082 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab81933-10eb-41b3-989a-6efee743f030 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:21.573479 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:21.657006 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:21.665858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04883258-7691-44a9-a8ac-e0ee7de4bf27 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:21.673417 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab81933-10eb-41b3-989a-6efee743f030 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:21.716287 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8da2bb-7111-426e-a6d0-a3afdcd48930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:21.717270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8da2bb-7111-426e-a6d0-a3afdcd48930 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:21.717854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8da2bb-7111-426e-a6d0-a3afdcd48930 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:21.718105 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57682936-7413-49a2-a3c7-92ea161de6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:21.718770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8da2bb-7111-426e-a6d0-a3afdcd48930 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:21.718892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57682936-7413-49a2-a3c7-92ea161de6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:21.719070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57682936-7413-49a2-a3c7-92ea161de6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:21.719370 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57682936-7413-49a2-a3c7-92ea161de6e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:21.806454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8da2bb-7111-426e-a6d0-a3afdcd48930 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:21.806911 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57682936-7413-49a2-a3c7-92ea161de6e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:21.807936 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8da2bb-7111-426e-a6d0-a3afdcd48930 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:21.808388 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57682936-7413-49a2-a3c7-92ea161de6e7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:21.839548 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:21.918468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 97/200] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:21.919106 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:21.920143 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8da2bb-7111-426e-a6d0-a3afdcd48930 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:21.929545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 104/201] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:21.930412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57682936-7413-49a2-a3c7-92ea161de6e7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:22.028995 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8da2bb-7111-426e-a6d0-a3afdcd48930 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:22.046315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57682936-7413-49a2-a3c7-92ea161de6e7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:22.060693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:22.067895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb15fe-97e6-4b1c-bcce-3ae859692153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:22.068504 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb15fe-97e6-4b1c-bcce-3ae859692153 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:22.068779 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb15fe-97e6-4b1c-bcce-3ae859692153 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:22.069060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb15fe-97e6-4b1c-bcce-3ae859692153 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:22.089096 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243c4f9-ee3c-4e78-80b2-b97b410b8ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:22.089979 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243c4f9-ee3c-4e78-80b2-b97b410b8ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:22.090401 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243c4f9-ee3c-4e78-80b2-b97b410b8ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:22.090853 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243c4f9-ee3c-4e78-80b2-b97b410b8ff7 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:22.092223 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243c4f9-ee3c-4e78-80b2-b97b410b8ff7 None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:22.094596 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243c4f9-ee3c-4e78-80b2-b97b410b8ff7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:22.096633 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 105/202] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:21 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:22.097625 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243c4f9-ee3c-4e78-80b2-b97b410b8ff7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:22.142329 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb15fe-97e6-4b1c-bcce-3ae859692153 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:22.143769 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb15fe-97e6-4b1c-bcce-3ae859692153 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:22.172540 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:22.210374 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243c4f9-ee3c-4e78-80b2-b97b410b8ff7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:22.261488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 98/203] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:22.261488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb15fe-97e6-4b1c-bcce-3ae859692153 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:22.262419 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dd6d7-8e2f-464b-bfd9-376cda13f95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:22.263499 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dd6d7-8e2f-464b-bfd9-376cda13f95a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:22.263945 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dd6d7-8e2f-464b-bfd9-376cda13f95a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:22.264360 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dd6d7-8e2f-464b-bfd9-376cda13f95a None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:22.266926 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dd6d7-8e2f-464b-bfd9-376cda13f95a None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:22.269340 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dd6d7-8e2f-464b-bfd9-376cda13f95a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:22.277285 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 106/204] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:22 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:22.278641 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dd6d7-8e2f-464b-bfd9-376cda13f95a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:22.290972 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:22.374999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb15fe-97e6-4b1c-bcce-3ae859692153 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:22.413947 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dd6d7-8e2f-464b-bfd9-376cda13f95a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:22.415189 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7f280-0c48-4f90-bb26-3f73357a62ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:22.415849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7f280-0c48-4f90-bb26-3f73357a62ea None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:22.416169 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7f280-0c48-4f90-bb26-3f73357a62ea None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:22.416440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7f280-0c48-4f90-bb26-3f73357a62ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:22.417897 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7f280-0c48-4f90-bb26-3f73357a62ea None admin] RBAC: Adding query filter params (role.id=f47755365f24477f9846225c9e91de2d, scope.project.id=b4d0c8983a01433eb5c05fa508bede98, user.id=bfb2ee6a3afe43c78883d3930589562c) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:22.417897 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7f280-0c48-4f90-bb26-3f73357a62ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:22.419039 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7f280-0c48-4f90-bb26-3f73357a62ea None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:22.424871 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 99/205] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:22 2022] GET /identity/v3/role_assignments?user.id=bfb2ee6a3afe43c78883d3930589562c&scope.project.id=b4d0c8983a01433eb5c05fa508bede98&role.id=f47755365f24477f9846225c9e91de2d => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:22.425686 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7f280-0c48-4f90-bb26-3f73357a62ea None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:22.455569 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc2964-ce58-443d-9be6-a07645b5025a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:22.456468 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc2964-ce58-443d-9be6-a07645b5025a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:22.456934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc2964-ce58-443d-9be6-a07645b5025a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:22.457444 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc2964-ce58-443d-9be6-a07645b5025a None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:22.458981 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc2964-ce58-443d-9be6-a07645b5025a None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:22.460722 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc2964-ce58-443d-9be6-a07645b5025a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:22.466645 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 107/206] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:22 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:22.470353 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc2964-ce58-443d-9be6-a07645b5025a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:22.473563 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:22.533664 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:22.541551 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7f280-0c48-4f90-bb26-3f73357a62ea None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:22.581603 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f466c2-a5a7-4f36-be6a-f583e350e658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:22.582148 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f466c2-a5a7-4f36-be6a-f583e350e658 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:22.582427 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f466c2-a5a7-4f36-be6a-f583e350e658 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:22.582728 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f466c2-a5a7-4f36-be6a-f583e350e658 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:22.583583 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f466c2-a5a7-4f36-be6a-f583e350e658 None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=b4d0c8983a01433eb5c05fa508bede98, user.id=bfb2ee6a3afe43c78883d3930589562c) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:22.583930 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f466c2-a5a7-4f36-be6a-f583e350e658 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:22.585520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f466c2-a5a7-4f36-be6a-f583e350e658 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:22.588000 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc2964-ce58-443d-9be6-a07645b5025a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:22.591753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 100/207] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:22 2022] GET /identity/v3/role_assignments?user.id=bfb2ee6a3afe43c78883d3930589562c&scope.project.id=b4d0c8983a01433eb5c05fa508bede98&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:22.641873 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd7d3e-e2e7-49b5-bd30-3e3868e850df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:22.642406 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd7d3e-e2e7-49b5-bd30-3e3868e850df None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:22.642707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd7d3e-e2e7-49b5-bd30-3e3868e850df None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:22.643025 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd7d3e-e2e7-49b5-bd30-3e3868e850df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:22.643769 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd7d3e-e2e7-49b5-bd30-3e3868e850df None admin] RBAC: Adding query filter params (role.id=86e796f30478458d99f1d139c853a1db, scope.project.id=bca76726c98f48f482c676027eb393d2, user.id=b53182f546814164abd49c1611eca319) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:22.644070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd7d3e-e2e7-49b5-bd30-3e3868e850df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:22.645537 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd7d3e-e2e7-49b5-bd30-3e3868e850df None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:22.655806 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 108/208] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:22 2022] GET /identity/v3/role_assignments?user.id=b53182f546814164abd49c1611eca319&scope.project.id=bca76726c98f48f482c676027eb393d2&role.id=86e796f30478458d99f1d139c853a1db => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:23.260051 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32da5e-92f2-4969-820b-bc23f9114130 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:23.260547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32da5e-92f2-4969-820b-bc23f9114130 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:23.260622 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32da5e-92f2-4969-820b-bc23f9114130 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:23.262114 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 101/209] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:23.267460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98439ce4-6c35-4bd1-b9ec-5eb71311c71f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:23.267765 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98439ce4-6c35-4bd1-b9ec-5eb71311c71f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:23.268082 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98439ce4-6c35-4bd1-b9ec-5eb71311c71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:23.287023 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-98439ce4-6c35-4bd1-b9ec-5eb71311c71f None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:23.288184 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-98439ce4-6c35-4bd1-b9ec-5eb71311c71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:23.353633 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:23.364034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 109/210] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:23.365330 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a7411-51bc-4ec9-af08-849a64c174af None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:23.365618 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a7411-51bc-4ec9-af08-849a64c174af None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:23.365894 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a7411-51bc-4ec9-af08-849a64c174af None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:23.367440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 102/211] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:59:23.372783 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2dbf3-9092-4911-afd5-c7e96092908e None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:23.373064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2dbf3-9092-4911-afd5-c7e96092908e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:23.373392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2dbf3-9092-4911-afd5-c7e96092908e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:23.392675 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-77d2dbf3-9092-4911-afd5-c7e96092908e None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:23.393859 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-77d2dbf3-9092-4911-afd5-c7e96092908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:23.462411 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 110/212] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:23.647704 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a7411-51bc-4ec9-af08-849a64c174af None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:23.736172 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2dbf3-9092-4911-afd5-c7e96092908e None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:23.775562 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a7411-51bc-4ec9-af08-849a64c174af None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:23.813192 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ca269-76df-4bf8-9f0d-f5ca13bda479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:23.814095 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ca269-76df-4bf8-9f0d-f5ca13bda479 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:23.814374 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ca269-76df-4bf8-9f0d-f5ca13bda479 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:23.814675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ca269-76df-4bf8-9f0d-f5ca13bda479 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:23.815789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ca269-76df-4bf8-9f0d-f5ca13bda479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:23.817402 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ca269-76df-4bf8-9f0d-f5ca13bda479 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:23.820108 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 103/213] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:59:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:23.825833 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ca269-76df-4bf8-9f0d-f5ca13bda479 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:23.855258 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2dbf3-9092-4911-afd5-c7e96092908e None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:23.870148 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:23.884297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:23.894373 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15ff9f-3730-4d77-b0ad-823525135bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:23.895273 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15ff9f-3730-4d77-b0ad-823525135bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:23.895556 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15ff9f-3730-4d77-b0ad-823525135bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:23.895931 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15ff9f-3730-4d77-b0ad-823525135bf8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:23.908227 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15ff9f-3730-4d77-b0ad-823525135bf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-6c15ff9f-3730-4d77-b0ad-823525135bf8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:23.914930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:23.916068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:23.916976 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:23.916976 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:23.916976 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:23.916976 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:23.916976 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:23.916976 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:23.916976 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:23.916976 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:23.916976 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:23.916976 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:23.916976 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:23.916976 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:23.916976 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 111/214] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:23.923787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15ff9f-3730-4d77-b0ad-823525135bf8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:23.952733 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ca269-76df-4bf8-9f0d-f5ca13bda479 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:24.008998 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e27558-2af6-4c88-abb4-3c4731922b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:24.011355 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e27558-2af6-4c88-abb4-3c4731922b3a None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:24.011355 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e27558-2af6-4c88-abb4-3c4731922b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:24.011355 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e27558-2af6-4c88-abb4-3c4731922b3a None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:24.011695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e27558-2af6-4c88-abb4-3c4731922b3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:24.013484 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e27558-2af6-4c88-abb4-3c4731922b3a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:24.056451 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 104/215] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 05:59:23 2022] POST /identity/v3/users => generated 297 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 05:59:24.073046 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15ff9f-3730-4d77-b0ad-823525135bf8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:24.117245 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1617ed2-2be8-4eba-a5a3-40711de11cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:24.118459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1617ed2-2be8-4eba-a5a3-40711de11cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:24.118944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1617ed2-2be8-4eba-a5a3-40711de11cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:24.119403 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1617ed2-2be8-4eba-a5a3-40711de11cd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:24.120411 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1617ed2-2be8-4eba-a5a3-40711de11cd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:24.120911 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1617ed2-2be8-4eba-a5a3-40711de11cd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:24.122645 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1617ed2-2be8-4eba-a5a3-40711de11cd0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:24.134017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 112/216] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:24.140262 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e27558-2af6-4c88-abb4-3c4731922b3a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:24.239379 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:24.259329 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e27558-2af6-4c88-abb4-3c4731922b3a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:24.299193 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e34b4-ab43-45c6-9998-4e923f0662cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:24.299908 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e34b4-ab43-45c6-9998-4e923f0662cc None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:24.300182 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e34b4-ab43-45c6-9998-4e923f0662cc None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:24.300403 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e34b4-ab43-45c6-9998-4e923f0662cc None admin] PATH_INFO: `/v3/users/e436644b5d244de1b0f7ccef620f9fed` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:24.317246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e34b4-ab43-45c6-9998-4e923f0662cc None admin] RBAC: Authorizing `identity:get_user(user_id=e436644b5d244de1b0f7ccef620f9fed)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:24.318749 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e34b4-ab43-45c6-9998-4e923f0662cc None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:24.320459 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 105/217] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:24 2022] GET /identity/v3/users/e436644b5d244de1b0f7ccef620f9fed => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:24.326345 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1617ed2-2be8-4eba-a5a3-40711de11cd0 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:24.386701 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:24.456501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1617ed2-2be8-4eba-a5a3-40711de11cd0 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:24.493356 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e39f3-6a10-42c8-b68f-56adcb63ffb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:24.493896 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e39f3-6a10-42c8-b68f-56adcb63ffb8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:24.494162 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e39f3-6a10-42c8-b68f-56adcb63ffb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:24.494428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e39f3-6a10-42c8-b68f-56adcb63ffb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:24.540722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7e9fe-6eac-4232-87be-37d303213d6c None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:24.541023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7e9fe-6eac-4232-87be-37d303213d6c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:24.541259 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7e9fe-6eac-4232-87be-37d303213d6c None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:24.543277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 106/218] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:24.550056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb3e8e-b899-41fa-8399-c5af9e1f3bc2 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:24.550622 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb3e8e-b899-41fa-8399-c5af9e1f3bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:24.550893 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb3e8e-b899-41fa-8399-c5af9e1f3bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:24.574834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-18bb3e8e-b899-41fa-8399-c5af9e1f3bc2 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:24.576021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-18bb3e8e-b899-41fa-8399-c5af9e1f3bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:24.576330 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e39f3-6a10-42c8-b68f-56adcb63ffb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:24.578481 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e39f3-6a10-42c8-b68f-56adcb63ffb8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:24.645852 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:24.675751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 107/219] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:59:24.707240 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 113/220] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:24.712890 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb3e8e-b899-41fa-8399-c5af9e1f3bc2 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:24.822648 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:24.866724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb3e8e-b899-41fa-8399-c5af9e1f3bc2 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:24.918135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c03be-e654-43dd-bcf9-f3cd49ff99de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:24.919181 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c03be-e654-43dd-bcf9-f3cd49ff99de None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:24.919669 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c03be-e654-43dd-bcf9-f3cd49ff99de None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:24.920140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c03be-e654-43dd-bcf9-f3cd49ff99de None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:24.922612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c03be-e654-43dd-bcf9-f3cd49ff99de None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:24.925181 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c03be-e654-43dd-bcf9-f3cd49ff99de None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:24.927821 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 108/221] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:24 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:24.934794 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e39f3-6a10-42c8-b68f-56adcb63ffb8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:24.978334 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c03be-e654-43dd-bcf9-f3cd49ff99de None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:25.095421 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c03be-e654-43dd-bcf9-f3cd49ff99de None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:25.163746 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e39f3-6a10-42c8-b68f-56adcb63ffb8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:25.166391 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239b832-cc63-4792-9908-393c6ffbb628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:25.167105 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239b832-cc63-4792-9908-393c6ffbb628 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:25.167395 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239b832-cc63-4792-9908-393c6ffbb628 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:25.167671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239b832-cc63-4792-9908-393c6ffbb628 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:25.184961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239b832-cc63-4792-9908-393c6ffbb628 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-f239b832-cc63-4792-9908-393c6ffbb628 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:25.190945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:25.191820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:25.192611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:25.192611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:25.192611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:25.192611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:25.192611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:25.192611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:25.192611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:25.192611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:25.192611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:25.192611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:25.192611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:25.192611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:25.195056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:25.195453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 109/222] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 05:59:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:59:25.200816 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f00ad3-c08e-4695-afdb-cda74c16f32d None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:25.201489 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f00ad3-c08e-4695-afdb-cda74c16f32d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:25.203112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f00ad3-c08e-4695-afdb-cda74c16f32d None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:25.205323 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 110/223] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:25.205898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f00ad3-c08e-4695-afdb-cda74c16f32d None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:25.219443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3097ffc-679e-4307-a5fc-905a142416e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:25.220940 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3097ffc-679e-4307-a5fc-905a142416e7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:25.221428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3097ffc-679e-4307-a5fc-905a142416e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:25.221894 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3097ffc-679e-4307-a5fc-905a142416e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:25.223409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3097ffc-679e-4307-a5fc-905a142416e7 None admin] RBAC: Adding query filter params (role.id=a2f35dad17e94553b63d07f8b8d8f1a4, scope.project.id=b4d0c8983a01433eb5c05fa508bede98, user.id=e436644b5d244de1b0f7ccef620f9fed) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:25.223922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3097ffc-679e-4307-a5fc-905a142416e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:25.226367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3097ffc-679e-4307-a5fc-905a142416e7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:25.235563 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 114/224] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:24 2022] GET /identity/v3/role_assignments?user.id=e436644b5d244de1b0f7ccef620f9fed&scope.project.id=b4d0c8983a01433eb5c05fa508bede98&role.id=a2f35dad17e94553b63d07f8b8d8f1a4 => generated 259 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:25.237101 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce975d1-7a1d-4264-859d-d5fce80e1f6d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:25.237584 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce975d1-7a1d-4264-859d-d5fce80e1f6d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:25.238074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce975d1-7a1d-4264-859d-d5fce80e1f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:25.260434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-8ce975d1-7a1d-4264-859d-d5fce80e1f6d None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:25.262271 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8ce975d1-7a1d-4264-859d-d5fce80e1f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:25.272801 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:25.371881 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 115/225] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:25.411162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f00ad3-c08e-4695-afdb-cda74c16f32d None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:25.450573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444d2fa-9b5b-4dac-84c0-0b370a85980b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:25.451583 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444d2fa-9b5b-4dac-84c0-0b370a85980b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:25.452654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444d2fa-9b5b-4dac-84c0-0b370a85980b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:25.453799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444d2fa-9b5b-4dac-84c0-0b370a85980b None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:25.454714 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444d2fa-9b5b-4dac-84c0-0b370a85980b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:25.455234 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444d2fa-9b5b-4dac-84c0-0b370a85980b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:25.456830 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444d2fa-9b5b-4dac-84c0-0b370a85980b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:25.466176 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 111/226] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 05:59:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:25.473113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce975d1-7a1d-4264-859d-d5fce80e1f6d None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:25.587949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce975d1-7a1d-4264-859d-d5fce80e1f6d None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:25.630462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef0ee2-2d14-44ed-82cd-ba5955235848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:25.631484 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef0ee2-2d14-44ed-82cd-ba5955235848 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:25.631979 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef0ee2-2d14-44ed-82cd-ba5955235848 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:25.632460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef0ee2-2d14-44ed-82cd-ba5955235848 None admin] PATH_INFO: `/v3/users/bfb2ee6a3afe43c78883d3930589562c` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:25.633840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef0ee2-2d14-44ed-82cd-ba5955235848 None admin] RBAC: Authorizing `identity:get_user(user_id=bfb2ee6a3afe43c78883d3930589562c)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:25.635541 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef0ee2-2d14-44ed-82cd-ba5955235848 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:25.641969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 116/227] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:25 2022] GET /identity/v3/users/bfb2ee6a3afe43c78883d3930589562c => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:25.644904 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444d2fa-9b5b-4dac-84c0-0b370a85980b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:25.713666 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:25.719374 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef0ee2-2d14-44ed-82cd-ba5955235848 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:25.736409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:25.776930 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444d2fa-9b5b-4dac-84c0-0b370a85980b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:25.818402 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fd15b-7ca8-496a-9322-3cee948d55d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:25.819401 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fd15b-7ca8-496a-9322-3cee948d55d7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:25.820306 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fd15b-7ca8-496a-9322-3cee948d55d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:25.820786 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fd15b-7ca8-496a-9322-3cee948d55d7 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:25.822162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140fd15b-7ca8-496a-9322-3cee948d55d7 None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:25.824001 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140fd15b-7ca8-496a-9322-3cee948d55d7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:25.830740 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 112/228] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:25 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:25.833311 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fd15b-7ca8-496a-9322-3cee948d55d7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:25.947817 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fd15b-7ca8-496a-9322-3cee948d55d7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:25.957112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef0ee2-2d14-44ed-82cd-ba5955235848 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:25.983711 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:25.989622 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebab87-8740-4202-87a1-3d86bed497b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:25.990716 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebab87-8740-4202-87a1-3d86bed497b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:25.991341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebab87-8740-4202-87a1-3d86bed497b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:25.991915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebab87-8740-4202-87a1-3d86bed497b0 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98/users/bfb2ee6a3afe43c78883d3930589562c/roles/f47755365f24477f9846225c9e91de2d` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:25.994625 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc181308-d397-4a61-a61d-2d5c089fe397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:25.995012 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebab87-8740-4202-87a1-3d86bed497b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4d0c8983a01433eb5c05fa508bede98, user_id=bfb2ee6a3afe43c78883d3930589562c, role_id=f47755365f24477f9846225c9e91de2d)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:25.995617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc181308-d397-4a61-a61d-2d5c089fe397 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:25.995958 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc181308-d397-4a61-a61d-2d5c089fe397 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:25.996292 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc181308-d397-4a61-a61d-2d5c089fe397 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:25.997521 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebab87-8740-4202-87a1-3d86bed497b0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:26.008077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc181308-d397-4a61-a61d-2d5c089fe397 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-cc181308-d397-4a61-a61d-2d5c089fe397 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:26.013395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:26.015840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:26.017776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:26.017776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:26.017776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:26.017776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:26.017776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:26.017776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:26.017776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:26.017776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:26.017776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:26.017776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:26.017776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:26.017776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:26.017776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 117/229] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 05:59:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:26.024708 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc181308-d397-4a61-a61d-2d5c089fe397 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:26.025110 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 113/230] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 05:59:25 2022] PUT /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98/users/bfb2ee6a3afe43c78883d3930589562c/roles/f47755365f24477f9846225c9e91de2d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 05:59:26.117445 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:26.153983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc181308-d397-4a61-a61d-2d5c089fe397 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:26.192221 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a602f9-d2b1-4add-81ac-dda672172e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:26.192831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a602f9-d2b1-4add-81ac-dda672172e81 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:26.193104 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a602f9-d2b1-4add-81ac-dda672172e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:26.193371 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a602f9-d2b1-4add-81ac-dda672172e81 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:26.194048 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a602f9-d2b1-4add-81ac-dda672172e81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:26.194345 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a602f9-d2b1-4add-81ac-dda672172e81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:26.196018 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a602f9-d2b1-4add-81ac-dda672172e81 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:26.206504 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 118/231] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:26.211873 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebab87-8740-4202-87a1-3d86bed497b0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:26.322141 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebab87-8740-4202-87a1-3d86bed497b0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:26.365588 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656b585-517f-4ee3-8632-d24030c03ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:26.366619 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656b585-517f-4ee3-8632-d24030c03ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:26.367070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656b585-517f-4ee3-8632-d24030c03ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:26.367779 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656b585-517f-4ee3-8632-d24030c03ad3 None admin] PATH_INFO: `/v3/users/bfb2ee6a3afe43c78883d3930589562c` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:26.369125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656b585-517f-4ee3-8632-d24030c03ad3 None admin] RBAC: Authorizing `identity:get_user(user_id=bfb2ee6a3afe43c78883d3930589562c)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:26.370876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656b585-517f-4ee3-8632-d24030c03ad3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:26.372709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 114/232] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:26 2022] GET /identity/v3/users/bfb2ee6a3afe43c78883d3930589562c => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:26.379057 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a602f9-d2b1-4add-81ac-dda672172e81 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:26.508229 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a602f9-d2b1-4add-81ac-dda672172e81 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:26.547463 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:26.559404 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0a66e-75ac-47a3-a486-0e67281a53db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:26.560368 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0a66e-75ac-47a3-a486-0e67281a53db None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:26.560853 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0a66e-75ac-47a3-a486-0e67281a53db None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:26.562034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0a66e-75ac-47a3-a486-0e67281a53db None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:26.563571 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0a66e-75ac-47a3-a486-0e67281a53db None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:26.565266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0a66e-75ac-47a3-a486-0e67281a53db None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:26.567339 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 119/233] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:26 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:26.573146 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656b585-517f-4ee3-8632-d24030c03ad3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:26.633301 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ddfaf-10de-48f0-a0cd-67bf61ee51a8 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:26.633610 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ddfaf-10de-48f0-a0cd-67bf61ee51a8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:26.633850 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ddfaf-10de-48f0-a0cd-67bf61ee51a8 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:26.635259 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 120/234] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:59:26.641298 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd410933-2e83-42bf-863e-562bb5a3f164 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:26.641565 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd410933-2e83-42bf-863e-562bb5a3f164 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:26.641836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd410933-2e83-42bf-863e-562bb5a3f164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:26.681395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-bd410933-2e83-42bf-863e-562bb5a3f164 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:26.683411 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bd410933-2e83-42bf-863e-562bb5a3f164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:26.714888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:26.778053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656b585-517f-4ee3-8632-d24030c03ad3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:26.803535 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 121/235] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:59:26.805924 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc09b9-c7ba-4430-85f1-63646708ca0f None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:26.806232 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc09b9-c7ba-4430-85f1-63646708ca0f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:26.806555 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc09b9-c7ba-4430-85f1-63646708ca0f None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:26.809004 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 122/236] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:26.815138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7541b-fbf4-4d33-9d97-4fc3f0fed259 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:26.815440 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7541b-fbf4-4d33-9d97-4fc3f0fed259 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:26.815752 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7541b-fbf4-4d33-9d97-4fc3f0fed259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:26.819648 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635b541-f524-4d4f-9cd4-882896039b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:26.820392 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635b541-f524-4d4f-9cd4-882896039b34 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:26.820737 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635b541-f524-4d4f-9cd4-882896039b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:26.821074 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635b541-f524-4d4f-9cd4-882896039b34 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98/users/bfb2ee6a3afe43c78883d3930589562c/roles/f6acb6fdca5a44de9ba00b2b0d4d3202` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:26.823262 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6635b541-f524-4d4f-9cd4-882896039b34 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4d0c8983a01433eb5c05fa508bede98, user_id=bfb2ee6a3afe43c78883d3930589562c, role_id=f6acb6fdca5a44de9ba00b2b0d4d3202)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:26.825701 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6635b541-f524-4d4f-9cd4-882896039b34 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:26.844537 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 115/237] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 05:59:26 2022] PUT /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98/users/bfb2ee6a3afe43c78883d3930589562c/roles/f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 05:59:26.848148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-dca7541b-fbf4-4d33-9d97-4fc3f0fed259 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:26.849490 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-dca7541b-fbf4-4d33-9d97-4fc3f0fed259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:26.942650 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 123/238] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:27.224655 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635b541-f524-4d4f-9cd4-882896039b34 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:27.285766 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7541b-fbf4-4d33-9d97-4fc3f0fed259 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:27.384734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635b541-f524-4d4f-9cd4-882896039b34 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:27.408002 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:27.419795 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:27.462092 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813d021-6670-45cb-9ea6-4c0cc119ef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:27.462784 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813d021-6670-45cb-9ea6-4c0cc119ef80 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:27.463068 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813d021-6670-45cb-9ea6-4c0cc119ef80 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:27.463337 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813d021-6670-45cb-9ea6-4c0cc119ef80 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:27.472495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7541b-fbf4-4d33-9d97-4fc3f0fed259 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:27.478040 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813d021-6670-45cb-9ea6-4c0cc119ef80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-0813d021-6670-45cb-9ea6-4c0cc119ef80 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:27.484724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:27.485606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:27.486367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:27.486367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:27.486367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:27.486367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:27.486367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:27.486367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:27.486367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:27.486367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:27.486367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:27.486367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:27.486367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:27.486367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:27.486367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 116/239] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:27.491659 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813d021-6670-45cb-9ea6-4c0cc119ef80 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:27.523079 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75216cb-cbb0-4c65-8cbc-a5275dfba87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:27.523820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75216cb-cbb0-4c65-8cbc-a5275dfba87c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:27.524094 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75216cb-cbb0-4c65-8cbc-a5275dfba87c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:27.524362 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75216cb-cbb0-4c65-8cbc-a5275dfba87c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:27.525467 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75216cb-cbb0-4c65-8cbc-a5275dfba87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:27.527043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75216cb-cbb0-4c65-8cbc-a5275dfba87c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:27.528628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 124/240] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:59:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:27.534776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75216cb-cbb0-4c65-8cbc-a5275dfba87c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:27.646547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813d021-6670-45cb-9ea6-4c0cc119ef80 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:27.713443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad09d7-17d1-44fb-919e-de3597c833dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:27.714061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad09d7-17d1-44fb-919e-de3597c833dd None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:27.714342 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad09d7-17d1-44fb-919e-de3597c833dd None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:27.714646 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad09d7-17d1-44fb-919e-de3597c833dd None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:27.715798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad09d7-17d1-44fb-919e-de3597c833dd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:27.716111 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad09d7-17d1-44fb-919e-de3597c833dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:27.718445 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad09d7-17d1-44fb-919e-de3597c833dd None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:27.718660 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75216cb-cbb0-4c65-8cbc-a5275dfba87c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:27.728564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 117/241] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:27.735593 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad09d7-17d1-44fb-919e-de3597c833dd None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:27.769634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:27.784537 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4fde2-c0db-4122-9298-fd8bf0146527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:27.785633 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4fde2-c0db-4122-9298-fd8bf0146527 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:27.786132 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4fde2-c0db-4122-9298-fd8bf0146527 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:27.786642 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4fde2-c0db-4122-9298-fd8bf0146527 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:27.787948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4fde2-c0db-4122-9298-fd8bf0146527 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:27.790041 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4fde2-c0db-4122-9298-fd8bf0146527 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:27.820030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:27.829607 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 125/242] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 05:59:27 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 05:59:27.945020 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad09d7-17d1-44fb-919e-de3597c833dd None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:27.997613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02b291-e124-4232-9fda-e012c8c037d0 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:27.997890 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02b291-e124-4232-9fda-e012c8c037d0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:27.998085 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02b291-e124-4232-9fda-e012c8c037d0 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:28.000078 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 126/243] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:59:28.005316 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485f7f0-6918-4cbe-bfb5-12e2e9cccd77 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:28.005593 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485f7f0-6918-4cbe-bfb5-12e2e9cccd77 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:28.005868 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485f7f0-6918-4cbe-bfb5-12e2e9cccd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:28.012028 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099dad8-1731-41e1-9282-b29df1106e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:28.012663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099dad8-1731-41e1-9282-b29df1106e76 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:28.012962 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099dad8-1731-41e1-9282-b29df1106e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:28.013232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099dad8-1731-41e1-9282-b29df1106e76 None admin] PATH_INFO: `/v3/users/e2ab9820ea64406b87296e33220f2e47` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:28.030734 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e485f7f0-6918-4cbe-bfb5-12e2e9cccd77 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:28.032008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099dad8-1731-41e1-9282-b29df1106e76 None admin] RBAC: Authorizing `identity:get_user(user_id=e2ab9820ea64406b87296e33220f2e47)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:28.032191 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e485f7f0-6918-4cbe-bfb5-12e2e9cccd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:28.033876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099dad8-1731-41e1-9282-b29df1106e76 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:28.041239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 118/244] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:27 2022] GET /identity/v3/users/e2ab9820ea64406b87296e33220f2e47 => generated 297 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:28.043524 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099dad8-1731-41e1-9282-b29df1106e76 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:28.120290 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 127/245] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:59:28.169907 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:28.179950 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099dad8-1731-41e1-9282-b29df1106e76 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:28.219288 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9ca58-92a0-4639-bdc1-34a6a753b2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:28.220005 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9ca58-92a0-4639-bdc1-34a6a753b2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:28.220282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9ca58-92a0-4639-bdc1-34a6a753b2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:28.220610 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9ca58-92a0-4639-bdc1-34a6a753b2b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:28.294959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9ca58-92a0-4639-bdc1-34a6a753b2b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:28.296468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9ca58-92a0-4639-bdc1-34a6a753b2b1 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:28.401787 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:28.411868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 119/246] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:28.417143 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485f7f0-6918-4cbe-bfb5-12e2e9cccd77 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:28.471458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9ca58-92a0-4639-bdc1-34a6a753b2b1 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:28.562451 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485f7f0-6918-4cbe-bfb5-12e2e9cccd77 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:28.579957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:28.605069 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32a3e3-80b6-4102-b015-94bee945b905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:28.605520 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32a3e3-80b6-4102-b015-94bee945b905 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:28.605793 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32a3e3-80b6-4102-b015-94bee945b905 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:28.606114 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32a3e3-80b6-4102-b015-94bee945b905 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:28.607304 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32a3e3-80b6-4102-b015-94bee945b905 None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:28.609280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32a3e3-80b6-4102-b015-94bee945b905 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:28.612044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 128/247] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:28 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:28.613270 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f07de-2346-4fff-bad0-da6d5d56acf7 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:28.613530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f07de-2346-4fff-bad0-da6d5d56acf7 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:28.613797 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f07de-2346-4fff-bad0-da6d5d56acf7 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:28.615414 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 129/248] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:28.617843 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f07de-2346-4fff-bad0-da6d5d56acf7 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:28.618810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9ca58-92a0-4639-bdc1-34a6a753b2b1 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:28.664858 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d426b-d8e8-4a4f-aecf-09e8defe1199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:28.666187 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d426b-d8e8-4a4f-aecf-09e8defe1199 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:28.666625 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d426b-d8e8-4a4f-aecf-09e8defe1199 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:28.667048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d426b-d8e8-4a4f-aecf-09e8defe1199 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:28.681842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d426b-d8e8-4a4f-aecf-09e8defe1199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-140d426b-d8e8-4a4f-aecf-09e8defe1199 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:28.687322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:28.688255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:28.689060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:28.689060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:28.689060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:28.689060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:28.689060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:28.689060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:28.689060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:28.689060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:28.689060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:28.689060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:28.689060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:28.689060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:28.689060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 120/249] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:59:28.690380 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45b269-c62c-4743-a879-8bcfd5ef851d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:28.695422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45b269-c62c-4743-a879-8bcfd5ef851d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:28.695422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45b269-c62c-4743-a879-8bcfd5ef851d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:28.716256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-2d45b269-c62c-4743-a879-8bcfd5ef851d None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:28.717749 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2d45b269-c62c-4743-a879-8bcfd5ef851d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:28.743867 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f07de-2346-4fff-bad0-da6d5d56acf7 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:28.776012 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:28.783652 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879c2d7-b8b3-43f2-b976-9f535f14356f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:28.784575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879c2d7-b8b3-43f2-b976-9f535f14356f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:28.785055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879c2d7-b8b3-43f2-b976-9f535f14356f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:28.785588 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879c2d7-b8b3-43f2-b976-9f535f14356f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:28.786477 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879c2d7-b8b3-43f2-b976-9f535f14356f None admin] RBAC: Adding query filter params (role.id=86e796f30478458d99f1d139c853a1db, scope.project.id=b4d0c8983a01433eb5c05fa508bede98, user.id=e2ab9820ea64406b87296e33220f2e47) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:28.786971 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879c2d7-b8b3-43f2-b976-9f535f14356f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:28.788778 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879c2d7-b8b3-43f2-b976-9f535f14356f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:28.796231 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 130/250] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:28 2022] GET /identity/v3/role_assignments?user.id=e2ab9820ea64406b87296e33220f2e47&scope.project.id=b4d0c8983a01433eb5c05fa508bede98&role.id=86e796f30478458d99f1d139c853a1db => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:28.797040 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 121/251] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:28.797601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879c2d7-b8b3-43f2-b976-9f535f14356f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:28.851205 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:28.911116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879c2d7-b8b3-43f2-b976-9f535f14356f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:28.965468 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812bea05-3843-40e9-847d-58eb79dce7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:28.966389 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bea05-3843-40e9-847d-58eb79dce7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:28.966879 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bea05-3843-40e9-847d-58eb79dce7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:28.967320 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812bea05-3843-40e9-847d-58eb79dce7e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:28.969185 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812bea05-3843-40e9-847d-58eb79dce7e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:28.969991 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812bea05-3843-40e9-847d-58eb79dce7e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:28.973309 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812bea05-3843-40e9-847d-58eb79dce7e8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:28.990682 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 131/252] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:28.994286 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45b269-c62c-4743-a879-8bcfd5ef851d None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:29.134633 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812bea05-3843-40e9-847d-58eb79dce7e8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:29.144495 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45b269-c62c-4743-a879-8bcfd5ef851d None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:29.222336 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf0958-d5dd-4dc4-a210-56113731b898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:29.223814 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf0958-d5dd-4dc4-a210-56113731b898 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:29.224100 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf0958-d5dd-4dc4-a210-56113731b898 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:29.224372 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf0958-d5dd-4dc4-a210-56113731b898 None admin] PATH_INFO: `/v3/users/e436644b5d244de1b0f7ccef620f9fed` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:29.226969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf0958-d5dd-4dc4-a210-56113731b898 None admin] RBAC: Authorizing `identity:get_user(user_id=e436644b5d244de1b0f7ccef620f9fed)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:29.228421 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf0958-d5dd-4dc4-a210-56113731b898 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:29.230043 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 122/253] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:28 2022] GET /identity/v3/users/e436644b5d244de1b0f7ccef620f9fed => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:29.236707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf0958-d5dd-4dc4-a210-56113731b898 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:29.302465 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812bea05-3843-40e9-847d-58eb79dce7e8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:29.317520 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:29.319677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:29.375343 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cbe78-928e-4cfd-8180-9e4ae8487e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:29.376558 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cbe78-928e-4cfd-8180-9e4ae8487e75 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:29.377905 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cbe78-928e-4cfd-8180-9e4ae8487e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:29.383850 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cbe78-928e-4cfd-8180-9e4ae8487e75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:29.411784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cbe78-928e-4cfd-8180-9e4ae8487e75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-637cbe78-928e-4cfd-8180-9e4ae8487e75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:29.419492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:29.421761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:29.427251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:29.427251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:29.427251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:29.427251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:29.427251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:29.427251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:29.427251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:29.427251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:29.427251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:29.427251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:29.427251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:29.427251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:29.432714 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 132/254] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 05:59:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:29.435047 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf0958-d5dd-4dc4-a210-56113731b898 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:29.444144 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba80e3-b890-4d95-ae1f-5d232bb88592 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:29.444732 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba80e3-b890-4d95-ae1f-5d232bb88592 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:29.445205 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba80e3-b890-4d95-ae1f-5d232bb88592 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:29.447052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 133/255] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:29.448318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba80e3-b890-4d95-ae1f-5d232bb88592 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:29.501141 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7ac54-1b35-410a-8a05-20f29269101b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:29.502160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7ac54-1b35-410a-8a05-20f29269101b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:29.502673 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7ac54-1b35-410a-8a05-20f29269101b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:29.503143 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7ac54-1b35-410a-8a05-20f29269101b None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:29.504501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7ac54-1b35-410a-8a05-20f29269101b None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:29.507757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7ac54-1b35-410a-8a05-20f29269101b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:29.509913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 123/256] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:29 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:29.514599 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc21a6-17ca-429f-b48c-54664b7eb7d9 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:29.514599 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc21a6-17ca-429f-b48c-54664b7eb7d9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:29.514599 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc21a6-17ca-429f-b48c-54664b7eb7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:29.551632 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d2fc21a6-17ca-429f-b48c-54664b7eb7d9 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:29.553088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d2fc21a6-17ca-429f-b48c-54664b7eb7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:29.591403 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba80e3-b890-4d95-ae1f-5d232bb88592 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:29.664116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c201b7-d9d3-4e8f-948e-31816cce270e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:29.667148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c201b7-d9d3-4e8f-948e-31816cce270e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:29.667613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c201b7-d9d3-4e8f-948e-31816cce270e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:29.668648 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c201b7-d9d3-4e8f-948e-31816cce270e None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:29.669993 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c201b7-d9d3-4e8f-948e-31816cce270e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:29.671807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c201b7-d9d3-4e8f-948e-31816cce270e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:29.673474 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c201b7-d9d3-4e8f-948e-31816cce270e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:29.691369 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 134/257] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 05:59:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:29.697147 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c201b7-d9d3-4e8f-948e-31816cce270e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:29.734958 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:29.748216 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 124/258] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:29.749766 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc21a6-17ca-429f-b48c-54664b7eb7d9 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:29.817340 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c201b7-d9d3-4e8f-948e-31816cce270e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:29.857059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d9939-2a85-475a-8417-ba45c21c0674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:29.858136 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d9939-2a85-475a-8417-ba45c21c0674 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:29.858859 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d9939-2a85-475a-8417-ba45c21c0674 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:29.859901 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d9939-2a85-475a-8417-ba45c21c0674 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98/users/e436644b5d244de1b0f7ccef620f9fed/roles/a2f35dad17e94553b63d07f8b8d8f1a4` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:29.862155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d9939-2a85-475a-8417-ba45c21c0674 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4d0c8983a01433eb5c05fa508bede98, user_id=e436644b5d244de1b0f7ccef620f9fed, role_id=a2f35dad17e94553b63d07f8b8d8f1a4)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:29.864044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d9939-2a85-475a-8417-ba45c21c0674 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:29.880444 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 135/259] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 05:59:29 2022] PUT /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98/users/e436644b5d244de1b0f7ccef620f9fed/roles/a2f35dad17e94553b63d07f8b8d8f1a4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 05:59:29.929519 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:30.020645 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc21a6-17ca-429f-b48c-54664b7eb7d9 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:30.069088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cde4e3-4d7f-4880-add8-6004955b655b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:30.069726 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cde4e3-4d7f-4880-add8-6004955b655b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:30.070008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cde4e3-4d7f-4880-add8-6004955b655b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:30.070282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cde4e3-4d7f-4880-add8-6004955b655b None admin] PATH_INFO: `/v3/users/bfb2ee6a3afe43c78883d3930589562c` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:30.075247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cde4e3-4d7f-4880-add8-6004955b655b None admin] RBAC: Authorizing `identity:get_user(user_id=bfb2ee6a3afe43c78883d3930589562c)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:30.077238 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cde4e3-4d7f-4880-add8-6004955b655b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:30.079311 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 125/260] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:29 2022] GET /identity/v3/users/bfb2ee6a3afe43c78883d3930589562c => generated 283 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:30.087176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d9939-2a85-475a-8417-ba45c21c0674 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:30.110217 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cde4e3-4d7f-4880-add8-6004955b655b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:30.227375 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d9939-2a85-475a-8417-ba45c21c0674 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:30.230856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:30.268763 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfc732-dda5-4723-aab7-025711a87a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:30.269616 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfc732-dda5-4723-aab7-025711a87a0b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:30.269999 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfc732-dda5-4723-aab7-025711a87a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:30.270376 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfc732-dda5-4723-aab7-025711a87a0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:30.320906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cde4e3-4d7f-4880-add8-6004955b655b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:30.349536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bfc732-dda5-4723-aab7-025711a87a0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:30.351684 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bfc732-dda5-4723-aab7-025711a87a0b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:30.363632 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:30.392212 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63346b9-90bf-4b64-b5e2-1863f9141962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:30.392973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63346b9-90bf-4b64-b5e2-1863f9141962 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:30.393256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63346b9-90bf-4b64-b5e2-1863f9141962 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:30.393529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63346b9-90bf-4b64-b5e2-1863f9141962 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:30.419590 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63346b9-90bf-4b64-b5e2-1863f9141962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-a63346b9-90bf-4b64-b5e2-1863f9141962 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:30.425609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:30.426503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:30.427351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:30.427351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:30.427351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:30.427351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:30.427351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:30.427351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:30.427351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:30.427351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:30.427351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:30.427351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:30.427351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:30.427351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:30.427351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 126/261] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 05:59:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:59:30.432544 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63346b9-90bf-4b64-b5e2-1863f9141962 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:30.472255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:30.491367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 136/262] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:30.497330 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfc732-dda5-4723-aab7-025711a87a0b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:30.602615 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63346b9-90bf-4b64-b5e2-1863f9141962 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:30.643305 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373fe8a-add5-4915-ad82-6d708b915d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:30.643924 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373fe8a-add5-4915-ad82-6d708b915d08 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:30.644202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373fe8a-add5-4915-ad82-6d708b915d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:30.644470 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373fe8a-add5-4915-ad82-6d708b915d08 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:30.645320 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373fe8a-add5-4915-ad82-6d708b915d08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:30.648724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373fe8a-add5-4915-ad82-6d708b915d08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:30.648724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373fe8a-add5-4915-ad82-6d708b915d08 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:30.658143 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 127/263] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:30.664447 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373fe8a-add5-4915-ad82-6d708b915d08 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:30.673105 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfc732-dda5-4723-aab7-025711a87a0b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:30.733886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65e358-8a1a-460c-b20e-5292cdfa936b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:30.734942 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65e358-8a1a-460c-b20e-5292cdfa936b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:30.735478 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65e358-8a1a-460c-b20e-5292cdfa936b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:30.735913 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65e358-8a1a-460c-b20e-5292cdfa936b None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:30.737286 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65e358-8a1a-460c-b20e-5292cdfa936b None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:30.743428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65e358-8a1a-460c-b20e-5292cdfa936b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:30.744327 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:30.746178 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 137/264] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:30 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:30.754484 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65e358-8a1a-460c-b20e-5292cdfa936b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:30.856529 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:30.860069 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373fe8a-add5-4915-ad82-6d708b915d08 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:30.928635 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65e358-8a1a-460c-b20e-5292cdfa936b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:30.939259 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a8e03-8c39-4117-803b-0b233078c573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:30.939923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a8e03-8c39-4117-803b-0b233078c573 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:30.940204 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a8e03-8c39-4117-803b-0b233078c573 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:30.940475 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a8e03-8c39-4117-803b-0b233078c573 None admin] PATH_INFO: `/v3/users/bfb2ee6a3afe43c78883d3930589562c` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:30.941676 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a8e03-8c39-4117-803b-0b233078c573 None admin] RBAC: Authorizing `identity:get_user(user_id=bfb2ee6a3afe43c78883d3930589562c)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:30.943158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a8e03-8c39-4117-803b-0b233078c573 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:30.945757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 128/265] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:30 2022] GET /identity/v3/users/bfb2ee6a3afe43c78883d3930589562c => generated 283 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:30.963626 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a8e03-8c39-4117-803b-0b233078c573 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:30.990428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde8f4d-aeb7-45e0-92fc-64e6a12316d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:30.991330 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde8f4d-aeb7-45e0-92fc-64e6a12316d6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:30.991656 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde8f4d-aeb7-45e0-92fc-64e6a12316d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:30.991931 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde8f4d-aeb7-45e0-92fc-64e6a12316d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:30.992985 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde8f4d-aeb7-45e0-92fc-64e6a12316d6 None admin] RBAC: Adding query filter params (role.id=f47755365f24477f9846225c9e91de2d, scope.project.id=b4d0c8983a01433eb5c05fa508bede98, user.id=bfb2ee6a3afe43c78883d3930589562c) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:30.993294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde8f4d-aeb7-45e0-92fc-64e6a12316d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:30.995029 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde8f4d-aeb7-45e0-92fc-64e6a12316d6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:31.019570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 138/266] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:30 2022] GET /identity/v3/role_assignments?user.id=bfb2ee6a3afe43c78883d3930589562c&scope.project.id=b4d0c8983a01433eb5c05fa508bede98&role.id=f47755365f24477f9846225c9e91de2d => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:31.115228 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eba5f-624e-406e-a156-dc7da5b4a308 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:31.115547 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eba5f-624e-406e-a156-dc7da5b4a308 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:31.115777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eba5f-624e-406e-a156-dc7da5b4a308 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:31.117266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 139/267] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:59:31.140235 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59120440-59ad-41f2-ab02-538d17f27b88 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:31.140456 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59120440-59ad-41f2-ab02-538d17f27b88 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:31.140687 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59120440-59ad-41f2-ab02-538d17f27b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:31.150276 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a8e03-8c39-4117-803b-0b233078c573 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:31.173851 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:31.198606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-59120440-59ad-41f2-ab02-538d17f27b88 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:31.199809 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-59120440-59ad-41f2-ab02-538d17f27b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:31.209165 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34f6a1-c4ac-405e-9c85-628d5b559c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:31.209795 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34f6a1-c4ac-405e-9c85-628d5b559c4b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:31.210075 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34f6a1-c4ac-405e-9c85-628d5b559c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:31.210348 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34f6a1-c4ac-405e-9c85-628d5b559c4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:31.322259 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34f6a1-c4ac-405e-9c85-628d5b559c4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:31.324653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34f6a1-c4ac-405e-9c85-628d5b559c4b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:31.409919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 140/268] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:31.482639 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 129/269] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:31.485081 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59120440-59ad-41f2-ab02-538d17f27b88 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:31.548515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:31.609531 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59120440-59ad-41f2-ab02-538d17f27b88 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:31.614981 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54644d-8dda-41f8-a806-2b666e7a971c None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:31.615265 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54644d-8dda-41f8-a806-2b666e7a971c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:31.615535 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54644d-8dda-41f8-a806-2b666e7a971c None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:31.617061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 130/270] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:59:31.624658 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704be58b-70a7-405c-b8b5-a781ac3f33ce None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:31.624658 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704be58b-70a7-405c-b8b5-a781ac3f33ce None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:31.624908 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704be58b-70a7-405c-b8b5-a781ac3f33ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:31.648748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-704be58b-70a7-405c-b8b5-a781ac3f33ce None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:31.650340 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-704be58b-70a7-405c-b8b5-a781ac3f33ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:31.664567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:31.684328 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa570256-e9aa-4d20-8f0d-918906097a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:31.685529 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa570256-e9aa-4d20-8f0d-918906097a45 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:31.685812 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa570256-e9aa-4d20-8f0d-918906097a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:31.686186 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa570256-e9aa-4d20-8f0d-918906097a45 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:31.687661 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa570256-e9aa-4d20-8f0d-918906097a45 None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:31.689842 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa570256-e9aa-4d20-8f0d-918906097a45 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:31.692016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 141/271] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:31 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:31.693426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6ea4a-4c94-40e8-a174-b24bede762c1 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:31.693773 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6ea4a-4c94-40e8-a174-b24bede762c1 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:31.694049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6ea4a-4c94-40e8-a174-b24bede762c1 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:31.695314 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 142/272] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:31.700315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6ea4a-4c94-40e8-a174-b24bede762c1 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:31.753871 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 131/273] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:59:31.755426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad40e6-eeac-4898-a192-da78a7b65546 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:31.755716 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad40e6-eeac-4898-a192-da78a7b65546 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:31.755985 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad40e6-eeac-4898-a192-da78a7b65546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:31.777663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-22ad40e6-eeac-4898-a192-da78a7b65546 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:31.778891 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-22ad40e6-eeac-4898-a192-da78a7b65546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:31.833173 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6ea4a-4c94-40e8-a174-b24bede762c1 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:31.858790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:31.873065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 132/274] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:31.874308 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad40e6-eeac-4898-a192-da78a7b65546 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:31.893467 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac6fd3-cb61-4eba-a10b-d1fbad750259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:31.894075 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac6fd3-cb61-4eba-a10b-d1fbad750259 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:31.894400 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac6fd3-cb61-4eba-a10b-d1fbad750259 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:31.894695 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac6fd3-cb61-4eba-a10b-d1fbad750259 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:31.895496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac6fd3-cb61-4eba-a10b-d1fbad750259 None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=b4d0c8983a01433eb5c05fa508bede98, user.id=bfb2ee6a3afe43c78883d3930589562c) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:31.895859 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac6fd3-cb61-4eba-a10b-d1fbad750259 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:31.898994 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac6fd3-cb61-4eba-a10b-d1fbad750259 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:31.910848 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 143/275] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:31 2022] GET /identity/v3/role_assignments?user.id=bfb2ee6a3afe43c78883d3930589562c&scope.project.id=b4d0c8983a01433eb5c05fa508bede98&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:31.989084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:32.031882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac6fd3-cb61-4eba-a10b-d1fbad750259 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:32.035768 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad40e6-eeac-4898-a192-da78a7b65546 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:32.083807 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f74fb5-07bd-45e0-9abe-2f6f7b427191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:32.084456 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f74fb5-07bd-45e0-9abe-2f6f7b427191 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:32.084732 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f74fb5-07bd-45e0-9abe-2f6f7b427191 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:32.085066 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f74fb5-07bd-45e0-9abe-2f6f7b427191 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:32.110547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f74fb5-07bd-45e0-9abe-2f6f7b427191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-59f74fb5-07bd-45e0-9abe-2f6f7b427191 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:32.118895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:32.121057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:32.123120 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:32.123120 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:32.123120 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:32.123120 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:32.123120 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:32.123120 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:32.123120 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:32.123120 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:32.123120 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:32.123120 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:32.123120 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:32.123120 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:32.123120 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 133/276] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:32.130136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f74fb5-07bd-45e0-9abe-2f6f7b427191 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:32.224004 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac6fd3-cb61-4eba-a10b-d1fbad750259 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:32.267321 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f74fb5-07bd-45e0-9abe-2f6f7b427191 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:32.269455 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b520977-1a04-460a-a447-0de883ff7d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:32.271218 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b520977-1a04-460a-a447-0de883ff7d36 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:32.271506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b520977-1a04-460a-a447-0de883ff7d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:32.271778 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b520977-1a04-460a-a447-0de883ff7d36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:32.273122 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b520977-1a04-460a-a447-0de883ff7d36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:32.275822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b520977-1a04-460a-a447-0de883ff7d36 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:32.279191 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 144/277] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:59:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:32.281117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b520977-1a04-460a-a447-0de883ff7d36 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:32.312555 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:32.319234 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7047f-16b1-492f-a081-5bd6099d8956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:32.319850 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7047f-16b1-492f-a081-5bd6099d8956 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:32.320123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7047f-16b1-492f-a081-5bd6099d8956 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:32.320389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7047f-16b1-492f-a081-5bd6099d8956 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:32.321057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7047f-16b1-492f-a081-5bd6099d8956 None admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:32.321896 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7047f-16b1-492f-a081-5bd6099d8956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:32.324078 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7047f-16b1-492f-a081-5bd6099d8956 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:32.333169 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 134/278] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:32.334125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7047f-16b1-492f-a081-5bd6099d8956 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:32.356504 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:32.411323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b520977-1a04-460a-a447-0de883ff7d36 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:32.448450 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7047f-16b1-492f-a081-5bd6099d8956 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:32.448945 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d42e20-6fd3-4c79-a620-7ed73f3a073a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:32.449570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d42e20-6fd3-4c79-a620-7ed73f3a073a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:32.449853 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d42e20-6fd3-4c79-a620-7ed73f3a073a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:32.450117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d42e20-6fd3-4c79-a620-7ed73f3a073a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:32.461921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d42e20-6fd3-4c79-a620-7ed73f3a073a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-f5d42e20-6fd3-4c79-a620-7ed73f3a073a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:32.468138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:32.469043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:32.469866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:32.469866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:32.469866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:32.469866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:32.469866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:32.469866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:32.469866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:32.469866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:32.469866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:32.469866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:32.469866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:32.469866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:32.474040 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 145/279] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:32.474982 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d42e20-6fd3-4c79-a620-7ed73f3a073a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:32.487119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8b9a1-a3da-43c9-ac84-92e252bd2ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:32.487734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8b9a1-a3da-43c9-ac84-92e252bd2ce8 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:32.488014 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8b9a1-a3da-43c9-ac84-92e252bd2ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:32.488284 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8b9a1-a3da-43c9-ac84-92e252bd2ce8 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:32.488926 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8b9a1-a3da-43c9-ac84-92e252bd2ce8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:32.490378 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8b9a1-a3da-43c9-ac84-92e252bd2ce8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:32.515360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 135/280] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 05:59:32 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 05:59:32.516203 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8b9a1-a3da-43c9-ac84-92e252bd2ce8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:32.582391 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:32.642871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d42e20-6fd3-4c79-a620-7ed73f3a073a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:32.656878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8b9a1-a3da-43c9-ac84-92e252bd2ce8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:32.665452 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:32.695271 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe02ca6-b9e3-4e27-803d-3bffbe5483b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:32.695908 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe02ca6-b9e3-4e27-803d-3bffbe5483b0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:32.696188 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe02ca6-b9e3-4e27-803d-3bffbe5483b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:32.696461 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe02ca6-b9e3-4e27-803d-3bffbe5483b0 None admin] PATH_INFO: `/v3/users/608953aeaf4a4d87bd819c8dd0d0f3ee` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:32.706296 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f12ee-872a-4231-b31b-f72ab4929b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:32.707388 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f12ee-872a-4231-b31b-f72ab4929b9f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:32.707667 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f12ee-872a-4231-b31b-f72ab4929b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:32.707936 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f12ee-872a-4231-b31b-f72ab4929b9f None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:32.708603 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f12ee-872a-4231-b31b-f72ab4929b9f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:32.708901 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f12ee-872a-4231-b31b-f72ab4929b9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:32.710308 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f12ee-872a-4231-b31b-f72ab4929b9f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:32.712263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe02ca6-b9e3-4e27-803d-3bffbe5483b0 None admin] RBAC: Authorizing `identity:get_user(user_id=608953aeaf4a4d87bd819c8dd0d0f3ee)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:32.713731 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe02ca6-b9e3-4e27-803d-3bffbe5483b0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:32.715837 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 146/281] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:32 2022] GET /identity/v3/users/608953aeaf4a4d87bd819c8dd0d0f3ee => generated 293 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:32.717742 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37163760-ea25-4ada-9287-5c44b3ff45a4 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:32.717742 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37163760-ea25-4ada-9287-5c44b3ff45a4 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:32.717936 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37163760-ea25-4ada-9287-5c44b3ff45a4 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:32.719452 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 147/282] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:32.722066 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 136/283] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:32.724560 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37163760-ea25-4ada-9287-5c44b3ff45a4 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:32.725219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62251c-5449-45b0-a65f-43c6592daf51 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:32.725499 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62251c-5449-45b0-a65f-43c6592daf51 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:32.725764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62251c-5449-45b0-a65f-43c6592daf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:32.745481 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8f62251c-5449-45b0-a65f-43c6592daf51 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:32.746577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8f62251c-5449-45b0-a65f-43c6592daf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:32.829722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 137/284] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:32.831024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62251c-5449-45b0-a65f-43c6592daf51 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:32.862978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37163760-ea25-4ada-9287-5c44b3ff45a4 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:32.890890 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:32.909759 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa46762-d4b2-4112-bb90-cd5146b7dba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:32.910376 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa46762-d4b2-4112-bb90-cd5146b7dba1 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:32.910726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa46762-d4b2-4112-bb90-cd5146b7dba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:32.910998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa46762-d4b2-4112-bb90-cd5146b7dba1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:32.912586 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa46762-d4b2-4112-bb90-cd5146b7dba1 None admin] RBAC: Adding query filter params (role.id=a2f35dad17e94553b63d07f8b8d8f1a4, user.id=608953aeaf4a4d87bd819c8dd0d0f3ee, scope.system=all) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:32.912900 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa46762-d4b2-4112-bb90-cd5146b7dba1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:32.914952 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa46762-d4b2-4112-bb90-cd5146b7dba1 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:32.923168 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 148/285] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Aug 23 05:59:32 2022] GET /identity/v3/role_assignments?user.id=608953aeaf4a4d87bd819c8dd0d0f3ee&scope.system=all&role.id=a2f35dad17e94553b63d07f8b8d8f1a4 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:32.954217 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62251c-5449-45b0-a65f-43c6592daf51 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:32.994980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3e9a9-be3e-48d5-9124-c0ce1c636258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:32.995626 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3e9a9-be3e-48d5-9124-c0ce1c636258 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:32.995905 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3e9a9-be3e-48d5-9124-c0ce1c636258 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:32.996172 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3e9a9-be3e-48d5-9124-c0ce1c636258 None admin] PATH_INFO: `/v3/users/e2ab9820ea64406b87296e33220f2e47` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:32.997311 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3e9a9-be3e-48d5-9124-c0ce1c636258 None admin] RBAC: Authorizing `identity:get_user(user_id=e2ab9820ea64406b87296e33220f2e47)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:32.999000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3e9a9-be3e-48d5-9124-c0ce1c636258 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:33.000764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 138/286] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:32 2022] GET /identity/v3/users/e2ab9820ea64406b87296e33220f2e47 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:33.006268 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa46762-d4b2-4112-bb90-cd5146b7dba1 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:33.070808 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:33.158688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3e9a9-be3e-48d5-9124-c0ce1c636258 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:33.175452 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa46762-d4b2-4112-bb90-cd5146b7dba1 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:33.225173 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ada5fd-f37b-4e82-a2d5-8263d0adb769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:33.226121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ada5fd-f37b-4e82-a2d5-8263d0adb769 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:33.226404 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ada5fd-f37b-4e82-a2d5-8263d0adb769 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:33.226701 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ada5fd-f37b-4e82-a2d5-8263d0adb769 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:33.228118 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ada5fd-f37b-4e82-a2d5-8263d0adb769 None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:33.230097 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ada5fd-f37b-4e82-a2d5-8263d0adb769 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:33.232509 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 149/287] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:33 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:33.239868 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ada5fd-f37b-4e82-a2d5-8263d0adb769 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:33.312532 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3e9a9-be3e-48d5-9124-c0ce1c636258 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:33.325654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:33.361262 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956b813-6187-4159-8b85-5ffdc9195af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:33.361964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956b813-6187-4159-8b85-5ffdc9195af7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:33.362243 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956b813-6187-4159-8b85-5ffdc9195af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:33.362586 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956b813-6187-4159-8b85-5ffdc9195af7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:33.375792 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956b813-6187-4159-8b85-5ffdc9195af7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-4956b813-6187-4159-8b85-5ffdc9195af7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:33.381235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:33.382295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:33.383155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:33.383155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:33.383155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:33.383155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:33.383155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:33.383155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:33.383155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:33.383155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:33.383155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:33.383155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:33.383155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:33.383155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:33.383155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 139/288] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:59:33.390584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956b813-6187-4159-8b85-5ffdc9195af7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:33.406615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ada5fd-f37b-4e82-a2d5-8263d0adb769 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:33.410633 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:33.477855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31afac7-3704-4496-8383-c430f3863119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:33.478446 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31afac7-3704-4496-8383-c430f3863119 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:33.478953 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31afac7-3704-4496-8383-c430f3863119 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:33.479231 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31afac7-3704-4496-8383-c430f3863119 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98/users/e2ab9820ea64406b87296e33220f2e47/roles/86e796f30478458d99f1d139c853a1db` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:33.485998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31afac7-3704-4496-8383-c430f3863119 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4d0c8983a01433eb5c05fa508bede98, user_id=e2ab9820ea64406b87296e33220f2e47, role_id=86e796f30478458d99f1d139c853a1db)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:33.487743 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31afac7-3704-4496-8383-c430f3863119 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:33.513913 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 150/289] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 05:59:33 2022] PUT /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98/users/e2ab9820ea64406b87296e33220f2e47/roles/86e796f30478458d99f1d139c853a1db => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 05:59:33.535895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956b813-6187-4159-8b85-5ffdc9195af7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:33.589950 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ff689-fbf8-4623-b945-24f55b988b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:33.590605 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ff689-fbf8-4623-b945-24f55b988b23 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:33.590888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ff689-fbf8-4623-b945-24f55b988b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:33.591156 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ff689-fbf8-4623-b945-24f55b988b23 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:33.591837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ff689-fbf8-4623-b945-24f55b988b23 None admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:33.592675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ff689-fbf8-4623-b945-24f55b988b23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:33.594258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ff689-fbf8-4623-b945-24f55b988b23 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:33.606932 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 140/290] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:33.612977 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31afac7-3704-4496-8383-c430f3863119 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:33.750978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31afac7-3704-4496-8383-c430f3863119 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:33.788108 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2a9bd-f072-4920-b2b9-5342fa6b35ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:33.788737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2a9bd-f072-4920-b2b9-5342fa6b35ca None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:33.789012 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2a9bd-f072-4920-b2b9-5342fa6b35ca None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:33.789276 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2a9bd-f072-4920-b2b9-5342fa6b35ca None admin] PATH_INFO: `/v3/users/e436644b5d244de1b0f7ccef620f9fed` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:33.790411 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2a9bd-f072-4920-b2b9-5342fa6b35ca None admin] RBAC: Authorizing `identity:get_user(user_id=e436644b5d244de1b0f7ccef620f9fed)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:33.791895 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2a9bd-f072-4920-b2b9-5342fa6b35ca None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:33.793550 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 151/291] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:33 2022] GET /identity/v3/users/e436644b5d244de1b0f7ccef620f9fed => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:33.799892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ff689-fbf8-4623-b945-24f55b988b23 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:33.835836 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:33.906669 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ff689-fbf8-4623-b945-24f55b988b23 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:33.940821 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99a9b1-66ef-4f49-b0f3-7e67e0863ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:33.941421 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99a9b1-66ef-4f49-b0f3-7e67e0863ffc None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:33.941697 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99a9b1-66ef-4f49-b0f3-7e67e0863ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:33.941963 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99a9b1-66ef-4f49-b0f3-7e67e0863ffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:34.046193 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99a9b1-66ef-4f49-b0f3-7e67e0863ffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:34.047609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99a9b1-66ef-4f49-b0f3-7e67e0863ffc None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:34.088517 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:34.196237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 141/292] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:34.201420 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2a9bd-f072-4920-b2b9-5342fa6b35ca None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:34.257824 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:34.352559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2a9bd-f072-4920-b2b9-5342fa6b35ca None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:34.401082 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500aeb27-9918-4ba7-a736-99f49bf813d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:34.401717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500aeb27-9918-4ba7-a736-99f49bf813d2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:34.401992 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500aeb27-9918-4ba7-a736-99f49bf813d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:34.402262 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500aeb27-9918-4ba7-a736-99f49bf813d2 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:34.403429 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500aeb27-9918-4ba7-a736-99f49bf813d2 None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:34.405080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500aeb27-9918-4ba7-a736-99f49bf813d2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:34.407077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 152/293] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:34 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:34.412744 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99a9b1-66ef-4f49-b0f3-7e67e0863ffc None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:34.534472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99a9b1-66ef-4f49-b0f3-7e67e0863ffc None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:34.572201 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:34.578957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b48cbe-92dd-4df0-9a72-67a75c501381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:34.579636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b48cbe-92dd-4df0-9a72-67a75c501381 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:34.579960 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b48cbe-92dd-4df0-9a72-67a75c501381 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:34.580231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b48cbe-92dd-4df0-9a72-67a75c501381 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:34.580966 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b48cbe-92dd-4df0-9a72-67a75c501381 None admin] RBAC: Adding query filter params (role.id=a2f35dad17e94553b63d07f8b8d8f1a4, scope.project.id=b4d0c8983a01433eb5c05fa508bede98, user.id=e436644b5d244de1b0f7ccef620f9fed) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:34.581268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b48cbe-92dd-4df0-9a72-67a75c501381 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:34.582729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b48cbe-92dd-4df0-9a72-67a75c501381 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:34.594076 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 142/294] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:34 2022] GET /identity/v3/role_assignments?user.id=e436644b5d244de1b0f7ccef620f9fed&scope.project.id=b4d0c8983a01433eb5c05fa508bede98&role.id=a2f35dad17e94553b63d07f8b8d8f1a4 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:35.031302 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804e074-824b-4e06-b124-47c415518240 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:35.031833 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804e074-824b-4e06-b124-47c415518240 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:35.031833 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804e074-824b-4e06-b124-47c415518240 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:35.033179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 153/295] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:35.038903 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00158d6-65c5-4144-b1c0-ed5533ddfc1d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:35.039357 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00158d6-65c5-4144-b1c0-ed5533ddfc1d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:35.039431 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00158d6-65c5-4144-b1c0-ed5533ddfc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:35.063394 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-f00158d6-65c5-4144-b1c0-ed5533ddfc1d None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:35.064666 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f00158d6-65c5-4144-b1c0-ed5533ddfc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:35.141785 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 143/296] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:35.168463 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f890d-de17-4e1e-8319-b0f012fc2079 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:35.168771 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f890d-de17-4e1e-8319-b0f012fc2079 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:35.168995 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f890d-de17-4e1e-8319-b0f012fc2079 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:35.171226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 154/297] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:35.176526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63644aa0-3047-474e-bf60-b01cd99d3cab None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:35.176808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63644aa0-3047-474e-bf60-b01cd99d3cab None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:35.177101 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63644aa0-3047-474e-bf60-b01cd99d3cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:35.195863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-63644aa0-3047-474e-bf60-b01cd99d3cab None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:35.197050 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-63644aa0-3047-474e-bf60-b01cd99d3cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:35.271061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 144/298] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:35.388601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddb0cd-1b4e-4460-8245-603075e178f8 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:35.389077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddb0cd-1b4e-4460-8245-603075e178f8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:35.389451 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddb0cd-1b4e-4460-8245-603075e178f8 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:35.391451 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 155/299] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:35.398101 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfce7c-b552-4dca-abe2-41e91c07dff2 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:35.398387 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfce7c-b552-4dca-abe2-41e91c07dff2 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:35.398678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfce7c-b552-4dca-abe2-41e91c07dff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:35.416985 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-56bfce7c-b552-4dca-abe2-41e91c07dff2 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:35.418163 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-56bfce7c-b552-4dca-abe2-41e91c07dff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:35.422751 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abddb0cd-1b4e-4460-8245-603075e178f8 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:35.473446 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:35.492142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 145/300] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:59:35.533708 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:35.539266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abddb0cd-1b4e-4460-8245-603075e178f8 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:35.575097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfce7c-b552-4dca-abe2-41e91c07dff2 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:35.575522 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de647042-8360-4018-a4b4-b01f8c692fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:35.576151 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de647042-8360-4018-a4b4-b01f8c692fbf None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:35.576488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de647042-8360-4018-a4b4-b01f8c692fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:35.576755 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de647042-8360-4018-a4b4-b01f8c692fbf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:35.588032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de647042-8360-4018-a4b4-b01f8c692fbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-de647042-8360-4018-a4b4-b01f8c692fbf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:35.593893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:35.594804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:35.595589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:35.595589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:35.595589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:35.595589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:35.595589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:35.595589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:35.595589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:35.595589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:35.595589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:35.595589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:35.595589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:35.595589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:35.595589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 156/301] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:59:35.602689 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de647042-8360-4018-a4b4-b01f8c692fbf None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:35.681398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfce7c-b552-4dca-abe2-41e91c07dff2 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:35.718296 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40986083-2319-4999-80a3-457e6fe2e116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:35.718953 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40986083-2319-4999-80a3-457e6fe2e116 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:35.719229 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40986083-2319-4999-80a3-457e6fe2e116 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:35.719496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40986083-2319-4999-80a3-457e6fe2e116 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:35.720619 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40986083-2319-4999-80a3-457e6fe2e116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:35.722059 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40986083-2319-4999-80a3-457e6fe2e116 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:35.728306 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 146/302] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:59:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:35.729172 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40986083-2319-4999-80a3-457e6fe2e116 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:35.739889 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de647042-8360-4018-a4b4-b01f8c692fbf None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:35.789878 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20570d-9a81-44c6-aea4-2ea3623bc016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:35.790886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20570d-9a81-44c6-aea4-2ea3623bc016 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:35.791927 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:35.792749 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20570d-9a81-44c6-aea4-2ea3623bc016 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:35.793220 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20570d-9a81-44c6-aea4-2ea3623bc016 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:35.794613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20570d-9a81-44c6-aea4-2ea3623bc016 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:35.795414 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20570d-9a81-44c6-aea4-2ea3623bc016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:35.798631 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20570d-9a81-44c6-aea4-2ea3623bc016 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:35.808117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 157/303] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:35.810552 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20570d-9a81-44c6-aea4-2ea3623bc016 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:35.834017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:35.848681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40986083-2319-4999-80a3-457e6fe2e116 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:35.888837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3b66e-368e-460e-a2fb-b93e2712cb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:35.890009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3b66e-368e-460e-a2fb-b93e2712cb6d None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:35.890292 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3b66e-368e-460e-a2fb-b93e2712cb6d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:35.890637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3b66e-368e-460e-a2fb-b93e2712cb6d None admin] PATH_INFO: `/v3/groups` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:35.891265 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a3b66e-368e-460e-a2fb-b93e2712cb6d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:35.892800 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a3b66e-368e-460e-a2fb-b93e2712cb6d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:35.913000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 147/304] 10.222.0.20 () {60 vars in 1134 bytes} [Tue Aug 23 05:59:35 2022] POST /identity/v3/groups => generated 229 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 05:59:35.914427 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3b66e-368e-460e-a2fb-b93e2712cb6d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:35.924214 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20570d-9a81-44c6-aea4-2ea3623bc016 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:35.978741 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876f467-c249-4d2a-b4ab-c70f74bf4f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:35.979384 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876f467-c249-4d2a-b4ab-c70f74bf4f32 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:35.979657 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876f467-c249-4d2a-b4ab-c70f74bf4f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:35.980216 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876f467-c249-4d2a-b4ab-c70f74bf4f32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:35.997072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876f467-c249-4d2a-b4ab-c70f74bf4f32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-6876f467-c249-4d2a-b4ab-c70f74bf4f32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:36.004233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:36.005256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:36.006159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:36.006159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:36.006159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:36.006159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:36.006159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:36.006159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:36.006159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:36.006159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:36.006159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:36.006159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:36.006159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:36.006159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:36.011105 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 158/305] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:36.012157 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876f467-c249-4d2a-b4ab-c70f74bf4f32 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:36.092993 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3b66e-368e-460e-a2fb-b93e2712cb6d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:36.107254 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:36.129833 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:36.137395 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96d21e-51ec-42f6-a70f-73e80bf4bf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:36.138031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96d21e-51ec-42f6-a70f-73e80bf4bf34 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:36.138306 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96d21e-51ec-42f6-a70f-73e80bf4bf34 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:36.138925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96d21e-51ec-42f6-a70f-73e80bf4bf34 None admin] PATH_INFO: `/v3/users/32bfde64b69e419db28c5daf0f7850ab` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:36.152021 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876f467-c249-4d2a-b4ab-c70f74bf4f32 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:36.153578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96d21e-51ec-42f6-a70f-73e80bf4bf34 None admin] RBAC: Authorizing `identity:get_user(user_id=32bfde64b69e419db28c5daf0f7850ab)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:36.155883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96d21e-51ec-42f6-a70f-73e80bf4bf34 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:36.162434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 148/306] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:35 2022] GET /identity/v3/users/32bfde64b69e419db28c5daf0f7850ab => generated 293 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:36.162434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfdbdd-7e79-4034-95b8-605eb5c6ccdb None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:36.162434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfdbdd-7e79-4034-95b8-605eb5c6ccdb None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:36.162434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfdbdd-7e79-4034-95b8-605eb5c6ccdb None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:36.162434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 149/307] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:36.163385 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfdbdd-7e79-4034-95b8-605eb5c6ccdb None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:36.193228 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277e88e-65b3-4e99-8556-d3b462e2582c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:36.193997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277e88e-65b3-4e99-8556-d3b462e2582c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:36.194280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277e88e-65b3-4e99-8556-d3b462e2582c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:36.194580 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277e88e-65b3-4e99-8556-d3b462e2582c None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:36.195265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277e88e-65b3-4e99-8556-d3b462e2582c None admin] RBAC: Adding query filter params (name=member) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:36.195554 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277e88e-65b3-4e99-8556-d3b462e2582c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:36.196950 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277e88e-65b3-4e99-8556-d3b462e2582c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:36.207674 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 159/308] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:36.213586 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c5706-637b-47dc-b90c-c2bb970c5f45 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:36.214327 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c5706-637b-47dc-b90c-c2bb970c5f45 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:36.214651 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c5706-637b-47dc-b90c-c2bb970c5f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:36.234879 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-592c5706-637b-47dc-b90c-c2bb970c5f45 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:36.236389 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-592c5706-637b-47dc-b90c-c2bb970c5f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:36.279266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfdbdd-7e79-4034-95b8-605eb5c6ccdb None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:36.310253 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 160/309] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:36.311728 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c5706-637b-47dc-b90c-c2bb970c5f45 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:36.321084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95afe228-036a-46e9-9e21-868a3cc88883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:36.321966 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afe228-036a-46e9-9e21-868a3cc88883 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:36.322437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afe228-036a-46e9-9e21-868a3cc88883 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:36.322996 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95afe228-036a-46e9-9e21-868a3cc88883 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:36.324068 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95afe228-036a-46e9-9e21-868a3cc88883 None admin] RBAC: Adding query filter params (role.id=86e796f30478458d99f1d139c853a1db, user.id=32bfde64b69e419db28c5daf0f7850ab, scope.system=all) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:36.324587 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95afe228-036a-46e9-9e21-868a3cc88883 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:36.326216 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95afe228-036a-46e9-9e21-868a3cc88883 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:36.332666 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 150/310] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Aug 23 05:59:36 2022] GET /identity/v3/role_assignments?user.id=32bfde64b69e419db28c5daf0f7850ab&scope.system=all&role.id=86e796f30478458d99f1d139c853a1db => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:36.406212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:36.441987 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c5706-637b-47dc-b90c-c2bb970c5f45 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:36.490709 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53d9db-9049-4e9d-a578-e46b99104a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:36.491347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53d9db-9049-4e9d-a578-e46b99104a3c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:36.491622 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53d9db-9049-4e9d-a578-e46b99104a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:36.491891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53d9db-9049-4e9d-a578-e46b99104a3c None admin] PATH_INFO: `/v3/users/608953aeaf4a4d87bd819c8dd0d0f3ee` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:36.493280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53d9db-9049-4e9d-a578-e46b99104a3c None admin] RBAC: Authorizing `identity:get_user(user_id=608953aeaf4a4d87bd819c8dd0d0f3ee)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:36.494739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53d9db-9049-4e9d-a578-e46b99104a3c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:36.496479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 161/311] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:36 2022] GET /identity/v3/users/608953aeaf4a4d87bd819c8dd0d0f3ee => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:36.502086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95afe228-036a-46e9-9e21-868a3cc88883 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:36.539714 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:36.614324 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95afe228-036a-46e9-9e21-868a3cc88883 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:36.624624 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53d9db-9049-4e9d-a578-e46b99104a3c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:36.655244 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862706b-285a-470b-8ef7-cb37e56863bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:36.656409 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862706b-285a-470b-8ef7-cb37e56863bd None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:36.656633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862706b-285a-470b-8ef7-cb37e56863bd None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:36.657149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862706b-285a-470b-8ef7-cb37e56863bd None admin] PATH_INFO: `/v3/system/users/608953aeaf4a4d87bd819c8dd0d0f3ee/roles/a2f35dad17e94553b63d07f8b8d8f1a4` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:36.659358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862706b-285a-470b-8ef7-cb37e56863bd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=608953aeaf4a4d87bd819c8dd0d0f3ee, role_id=a2f35dad17e94553b63d07f8b8d8f1a4)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:36.660748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862706b-285a-470b-8ef7-cb37e56863bd None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:36.674940 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 151/312] 10.222.0.20 () {58 vars in 1334 bytes} [Tue Aug 23 05:59:36 2022] PUT /identity/v3/system/users/608953aeaf4a4d87bd819c8dd0d0f3ee/roles/a2f35dad17e94553b63d07f8b8d8f1a4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 05:59:36.751892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53d9db-9049-4e9d-a578-e46b99104a3c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:36.809068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ec493-231a-4e42-8b8d-430b96f36719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:36.809862 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ec493-231a-4e42-8b8d-430b96f36719 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:36.810146 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ec493-231a-4e42-8b8d-430b96f36719 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:36.810416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ec493-231a-4e42-8b8d-430b96f36719 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:36.822806 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:36.827292 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ec493-231a-4e42-8b8d-430b96f36719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-f20ec493-231a-4e42-8b8d-430b96f36719 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:36.835177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:36.836070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:36.836851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:36.836851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:36.836851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:36.836851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:36.836851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:36.836851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:36.836851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:36.836851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:36.836851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:36.836851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:36.836851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:36.836851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:36.842875 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 162/313] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:59:36.845495 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862706b-285a-470b-8ef7-cb37e56863bd None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:36.954535 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:36.958957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862706b-285a-470b-8ef7-cb37e56863bd None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:36.995124 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2d25b-fd0e-4338-aded-3ae75cfec836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:36.995741 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2d25b-fd0e-4338-aded-3ae75cfec836 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:36.996022 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2d25b-fd0e-4338-aded-3ae75cfec836 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:36.996298 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2d25b-fd0e-4338-aded-3ae75cfec836 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:36.997031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2d25b-fd0e-4338-aded-3ae75cfec836 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:36.997335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2d25b-fd0e-4338-aded-3ae75cfec836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:36.999438 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2d25b-fd0e-4338-aded-3ae75cfec836 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:37.012094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 152/314] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:37.014352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ec493-231a-4e42-8b8d-430b96f36719 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:37.133771 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ec493-231a-4e42-8b8d-430b96f36719 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:37.188109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb049637-7832-4c29-a108-b726a52aa938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:37.189052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049637-7832-4c29-a108-b726a52aa938 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:37.189508 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049637-7832-4c29-a108-b726a52aa938 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:37.189957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049637-7832-4c29-a108-b726a52aa938 None admin] PATH_INFO: `/v3/users/e2ab9820ea64406b87296e33220f2e47` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:37.191376 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb049637-7832-4c29-a108-b726a52aa938 None admin] RBAC: Authorizing `identity:get_user(user_id=e2ab9820ea64406b87296e33220f2e47)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:37.193947 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb049637-7832-4c29-a108-b726a52aa938 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:37.201247 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 163/315] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:37 2022] GET /identity/v3/users/e2ab9820ea64406b87296e33220f2e47 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:37.203468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2d25b-fd0e-4338-aded-3ae75cfec836 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:37.324887 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2d25b-fd0e-4338-aded-3ae75cfec836 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:37.359943 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c999b-ed81-46c6-9134-63368c4c6b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:37.360554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c999b-ed81-46c6-9134-63368c4c6b9d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:37.360833 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c999b-ed81-46c6-9134-63368c4c6b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:37.361106 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c999b-ed81-46c6-9134-63368c4c6b9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:37.390077 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:37.431648 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c999b-ed81-46c6-9134-63368c4c6b9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:37.433126 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c999b-ed81-46c6-9134-63368c4c6b9d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:37.542814 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 153/316] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 05:59:37.551937 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb049637-7832-4c29-a108-b726a52aa938 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:37.622191 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:37.659134 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb049637-7832-4c29-a108-b726a52aa938 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:37.695665 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c0caa-a22e-49f1-bb82-5aa6f30b7db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:37.696650 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c0caa-a22e-49f1-bb82-5aa6f30b7db9 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:37.697124 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c0caa-a22e-49f1-bb82-5aa6f30b7db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:37.697591 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c0caa-a22e-49f1-bb82-5aa6f30b7db9 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:37.698970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c0caa-a22e-49f1-bb82-5aa6f30b7db9 None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:37.700622 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c0caa-a22e-49f1-bb82-5aa6f30b7db9 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:37.708130 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 164/317] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:37 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:37.709718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c999b-ed81-46c6-9134-63368c4c6b9d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:37.771010 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:37.816231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c999b-ed81-46c6-9134-63368c4c6b9d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:37.851680 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650402f7-d900-4617-a05f-2a0693be5131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:37.852739 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650402f7-d900-4617-a05f-2a0693be5131 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:37.853217 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650402f7-d900-4617-a05f-2a0693be5131 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:37.853749 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650402f7-d900-4617-a05f-2a0693be5131 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:37.854741 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650402f7-d900-4617-a05f-2a0693be5131 None admin] RBAC: Adding query filter params (role.id=86e796f30478458d99f1d139c853a1db, scope.project.id=b4d0c8983a01433eb5c05fa508bede98, user.id=e2ab9820ea64406b87296e33220f2e47) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:37.855262 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650402f7-d900-4617-a05f-2a0693be5131 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:37.856898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650402f7-d900-4617-a05f-2a0693be5131 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:37.863536 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 154/318] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 05:59:37 2022] GET /identity/v3/role_assignments?user.id=e2ab9820ea64406b87296e33220f2e47&scope.project.id=b4d0c8983a01433eb5c05fa508bede98&role.id=86e796f30478458d99f1d139c853a1db => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:38.018976 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69803663-1a3e-4076-9c2d-26d943f0f467 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:38.019314 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69803663-1a3e-4076-9c2d-26d943f0f467 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:38.019532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69803663-1a3e-4076-9c2d-26d943f0f467 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:38.021196 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 165/319] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:38.028243 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074729a-7b56-48d4-b987-cc48598c611a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:38.028813 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074729a-7b56-48d4-b987-cc48598c611a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:38.029413 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074729a-7b56-48d4-b987-cc48598c611a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:38.050351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-6074729a-7b56-48d4-b987-cc48598c611a None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:38.051610 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6074729a-7b56-48d4-b987-cc48598c611a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:38.128650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 155/320] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:38.396527 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69803663-1a3e-4076-9c2d-26d943f0f467 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:38.444578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eeef39-2be6-45dd-a146-b8f2ee42a0f2 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:38.445103 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eeef39-2be6-45dd-a146-b8f2ee42a0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:38.445366 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eeef39-2be6-45dd-a146-b8f2ee42a0f2 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:38.446848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 156/321] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:38.453970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90275f08-f4cb-4d3d-861b-0bf04643f96b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:38.454256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90275f08-f4cb-4d3d-861b-0bf04643f96b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:38.454533 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90275f08-f4cb-4d3d-861b-0bf04643f96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:38.473147 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-90275f08-f4cb-4d3d-861b-0bf04643f96b None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:38.474416 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-90275f08-f4cb-4d3d-861b-0bf04643f96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:38.513731 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69803663-1a3e-4076-9c2d-26d943f0f467 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:38.530219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:38.552166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de853d9-84f1-4bad-9806-ebd50b2a035a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:38.552612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 157/322] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:38.558636 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de853d9-84f1-4bad-9806-ebd50b2a035a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:38.558636 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de853d9-84f1-4bad-9806-ebd50b2a035a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:38.558636 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de853d9-84f1-4bad-9806-ebd50b2a035a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:38.559343 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de853d9-84f1-4bad-9806-ebd50b2a035a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:38.563315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de853d9-84f1-4bad-9806-ebd50b2a035a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:38.565758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 166/323] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:59:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:38.572931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90275f08-f4cb-4d3d-861b-0bf04643f96b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:38.677821 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90275f08-f4cb-4d3d-861b-0bf04643f96b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:38.711813 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b98d66-8297-4601-a882-e28f1d3c8023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:38.712418 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b98d66-8297-4601-a882-e28f1d3c8023 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:38.712694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b98d66-8297-4601-a882-e28f1d3c8023 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:38.712961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b98d66-8297-4601-a882-e28f1d3c8023 None admin] PATH_INFO: `/v3/groups` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:38.713586 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b98d66-8297-4601-a882-e28f1d3c8023 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:38.715048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b98d66-8297-4601-a882-e28f1d3c8023 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:38.741082 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 158/324] 10.222.0.20 () {60 vars in 1134 bytes} [Tue Aug 23 05:59:38 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 05:59:38.772303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2afd1-dffe-4a50-80c9-a18d69a7272a None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:38.772569 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2afd1-dffe-4a50-80c9-a18d69a7272a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:38.772845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2afd1-dffe-4a50-80c9-a18d69a7272a None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:38.774370 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 167/325] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:38.780677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6390ae-d1ef-4a86-af4f-432e4a8369b5 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:38.781005 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6390ae-d1ef-4a86-af4f-432e4a8369b5 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:38.781213 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6390ae-d1ef-4a86-af4f-432e4a8369b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:38.802234 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ca6390ae-d1ef-4a86-af4f-432e4a8369b5 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:38.803480 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ca6390ae-d1ef-4a86-af4f-432e4a8369b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:38.839192 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2afd1-dffe-4a50-80c9-a18d69a7272a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:38.844784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:38.851041 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:38.890124 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 159/326] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:59:38.978649 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2afd1-dffe-4a50-80c9-a18d69a7272a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:39.018294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee09bae-d702-4bb5-8a02-4aaa87b76b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:39.019033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee09bae-d702-4bb5-8a02-4aaa87b76b70 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:39.019273 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee09bae-d702-4bb5-8a02-4aaa87b76b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:39.019546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee09bae-d702-4bb5-8a02-4aaa87b76b70 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:39.031720 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee09bae-d702-4bb5-8a02-4aaa87b76b70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-fee09bae-d702-4bb5-8a02-4aaa87b76b70 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:39.037396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:39.038261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:39.039109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:39.039109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:39.039109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:39.039109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:39.039109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:39.039109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:39.039109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:39.039109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:39.039109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:39.039109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:39.039109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:39.039109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:39.039109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 168/327] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:59:39.046284 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6390ae-d1ef-4a86-af4f-432e4a8369b5 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:39.165253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6390ae-d1ef-4a86-af4f-432e4a8369b5 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:39.165831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee09bae-d702-4bb5-8a02-4aaa87b76b70 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:39.213922 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd302a20-651a-4668-903c-053ce52cfc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:39.214461 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd302a20-651a-4668-903c-053ce52cfc8b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:39.214781 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd302a20-651a-4668-903c-053ce52cfc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:39.215049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd302a20-651a-4668-903c-053ce52cfc8b None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:39.215714 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd302a20-651a-4668-903c-053ce52cfc8b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:39.216009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd302a20-651a-4668-903c-053ce52cfc8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:39.217742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd302a20-651a-4668-903c-053ce52cfc8b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:39.226790 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 160/328] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:39.232406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd302a20-651a-4668-903c-053ce52cfc8b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:39.250068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:39.270860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:39.297993 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee09bae-d702-4bb5-8a02-4aaa87b76b70 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:39.339128 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd302a20-651a-4668-903c-053ce52cfc8b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:39.340587 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd205713-9975-40f9-8d3f-00effe03d2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:39.341505 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd205713-9975-40f9-8d3f-00effe03d2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:39.342434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd205713-9975-40f9-8d3f-00effe03d2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:39.343008 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd205713-9975-40f9-8d3f-00effe03d2e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:39.356245 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd205713-9975-40f9-8d3f-00effe03d2e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-bd205713-9975-40f9-8d3f-00effe03d2e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:39.362770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:39.363863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:39.364831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:39.364831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:39.364831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:39.364831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:39.364831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:39.364831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:39.364831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:39.364831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:39.364831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:39.364831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:39.364831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:39.364831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:39.367239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 169/329] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:59:39.376232 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd205713-9975-40f9-8d3f-00effe03d2e5 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:39.382167 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d8543-90d9-4bb4-bc35-b728d4db0556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:39.383092 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d8543-90d9-4bb4-bc35-b728d4db0556 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:39.383563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d8543-90d9-4bb4-bc35-b728d4db0556 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:39.384050 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d8543-90d9-4bb4-bc35-b728d4db0556 None admin] PATH_INFO: `/v3/users/32bfde64b69e419db28c5daf0f7850ab` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:39.385433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d8543-90d9-4bb4-bc35-b728d4db0556 None admin] RBAC: Authorizing `identity:get_user(user_id=32bfde64b69e419db28c5daf0f7850ab)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:39.387148 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d8543-90d9-4bb4-bc35-b728d4db0556 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:39.389065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 161/330] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:39 2022] GET /identity/v3/users/32bfde64b69e419db28c5daf0f7850ab => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:39.394949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d8543-90d9-4bb4-bc35-b728d4db0556 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:39.505389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d8543-90d9-4bb4-bc35-b728d4db0556 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:39.511759 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:39.526687 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd205713-9975-40f9-8d3f-00effe03d2e5 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:39.557753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf01c38-c997-4b14-8d32-2b2b3e6d7da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:39.558454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf01c38-c997-4b14-8d32-2b2b3e6d7da3 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:39.558982 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf01c38-c997-4b14-8d32-2b2b3e6d7da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:39.559454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf01c38-c997-4b14-8d32-2b2b3e6d7da3 None admin] PATH_INFO: `/v3/system/users/32bfde64b69e419db28c5daf0f7850ab/roles/86e796f30478458d99f1d139c853a1db` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:39.561971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf01c38-c997-4b14-8d32-2b2b3e6d7da3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=32bfde64b69e419db28c5daf0f7850ab, role_id=86e796f30478458d99f1d139c853a1db)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:39.563609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf01c38-c997-4b14-8d32-2b2b3e6d7da3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:39.566912 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:39.576232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 162/331] 10.222.0.20 () {58 vars in 1334 bytes} [Tue Aug 23 05:59:39 2022] PUT /identity/v3/system/users/32bfde64b69e419db28c5daf0f7850ab/roles/86e796f30478458d99f1d139c853a1db => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 05:59:39.583277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552709b-1306-4b6f-851c-ce4df3d8c448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:39.584332 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552709b-1306-4b6f-851c-ce4df3d8c448 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:39.584677 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552709b-1306-4b6f-851c-ce4df3d8c448 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:39.584954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552709b-1306-4b6f-851c-ce4df3d8c448 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:39.586287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552709b-1306-4b6f-851c-ce4df3d8c448 None admin] RBAC: Adding query filter params (name=member) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:39.586665 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552709b-1306-4b6f-851c-ce4df3d8c448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:39.588080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552709b-1306-4b6f-851c-ce4df3d8c448 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:39.605270 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 170/332] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:39.611657 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf01c38-c997-4b14-8d32-2b2b3e6d7da3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:39.749286 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf01c38-c997-4b14-8d32-2b2b3e6d7da3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:39.793885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78aed32-9980-43ba-8d93-77af789813f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:39.794505 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78aed32-9980-43ba-8d93-77af789813f0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:39.794804 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78aed32-9980-43ba-8d93-77af789813f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:39.795172 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78aed32-9980-43ba-8d93-77af789813f0 None admin] PATH_INFO: `/v3/users/608953aeaf4a4d87bd819c8dd0d0f3ee` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:39.796685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78aed32-9980-43ba-8d93-77af789813f0 None admin] RBAC: Authorizing `identity:get_user(user_id=608953aeaf4a4d87bd819c8dd0d0f3ee)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:39.798261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78aed32-9980-43ba-8d93-77af789813f0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:39.800322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 163/333] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:39 2022] GET /identity/v3/users/608953aeaf4a4d87bd819c8dd0d0f3ee => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:39.805928 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552709b-1306-4b6f-851c-ce4df3d8c448 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:39.922020 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552709b-1306-4b6f-851c-ce4df3d8c448 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:39.968002 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d299dcf-de05-4bfe-a2f3-f3ad3990c8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:39.968647 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d299dcf-de05-4bfe-a2f3-f3ad3990c8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:39.968937 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d299dcf-de05-4bfe-a2f3-f3ad3990c8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:39.969271 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d299dcf-de05-4bfe-a2f3-f3ad3990c8d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:39.970011 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d299dcf-de05-4bfe-a2f3-f3ad3990c8d3 None admin] RBAC: Adding query filter params (role.id=a2f35dad17e94553b63d07f8b8d8f1a4, user.id=608953aeaf4a4d87bd819c8dd0d0f3ee, scope.system=all) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:39.970315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d299dcf-de05-4bfe-a2f3-f3ad3990c8d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:39.971824 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d299dcf-de05-4bfe-a2f3-f3ad3990c8d3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:39.985453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 171/334] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Aug 23 05:59:39 2022] GET /identity/v3/role_assignments?user.id=608953aeaf4a4d87bd819c8dd0d0f3ee&scope.system=all&role.id=a2f35dad17e94553b63d07f8b8d8f1a4 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:41.109237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481862a-1ef4-4d13-9bb2-5643b2a6971f None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:41.110338 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481862a-1ef4-4d13-9bb2-5643b2a6971f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:41.110876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481862a-1ef4-4d13-9bb2-5643b2a6971f None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:41.112910 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 164/335] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:41.119749 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462e7fb-c5a3-4470-848b-97409ba5b646 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:41.120750 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462e7fb-c5a3-4470-848b-97409ba5b646 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:41.121221 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462e7fb-c5a3-4470-848b-97409ba5b646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:41.143696 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c462e7fb-c5a3-4470-848b-97409ba5b646 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:41.145396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c462e7fb-c5a3-4470-848b-97409ba5b646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:41.191002 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:41.225672 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 172/336] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:59:41.519543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481862a-1ef4-4d13-9bb2-5643b2a6971f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:41.558590 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:41.644802 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481862a-1ef4-4d13-9bb2-5643b2a6971f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:41.691181 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7094a-a1e3-48ce-b26f-ed0805bf588f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:41.691826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7094a-a1e3-48ce-b26f-ed0805bf588f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:41.692127 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7094a-a1e3-48ce-b26f-ed0805bf588f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:41.692398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7094a-a1e3-48ce-b26f-ed0805bf588f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:41.705614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7094a-a1e3-48ce-b26f-ed0805bf588f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-b1c7094a-a1e3-48ce-b26f-ed0805bf588f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:41.710954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:41.711826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:41.713019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:41.713019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:41.713019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:41.713019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:41.713019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:41.713019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:41.713019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:41.713019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:41.713019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:41.713019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:41.713019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:41.713019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:41.713019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 165/337] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:41.718552 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462e7fb-c5a3-4470-848b-97409ba5b646 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:41.858255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462e7fb-c5a3-4470-848b-97409ba5b646 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:41.898880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe1112-0e7c-4ee5-9658-dd46a5dd39f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:41.899580 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe1112-0e7c-4ee5-9658-dd46a5dd39f3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:41.899856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe1112-0e7c-4ee5-9658-dd46a5dd39f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:41.900120 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe1112-0e7c-4ee5-9658-dd46a5dd39f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:41.900795 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe1112-0e7c-4ee5-9658-dd46a5dd39f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:41.901118 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe1112-0e7c-4ee5-9658-dd46a5dd39f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:41.902775 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe1112-0e7c-4ee5-9658-dd46a5dd39f3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:41.919239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 173/338] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:41.925998 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7094a-a1e3-48ce-b26f-ed0805bf588f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:41.939846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0f91a-545f-4ceb-8cb7-a4de2a435db2 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:41.940150 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0f91a-545f-4ceb-8cb7-a4de2a435db2 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:41.940390 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0f91a-545f-4ceb-8cb7-a4de2a435db2 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:41.941954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 174/339] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:59:41.953607 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0325746-37a4-4fcf-b287-2d08e4ef631c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:41.953879 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0325746-37a4-4fcf-b287-2d08e4ef631c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:41.954312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0325746-37a4-4fcf-b287-2d08e4ef631c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:41.983723 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c0325746-37a4-4fcf-b287-2d08e4ef631c None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:41.985256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c0325746-37a4-4fcf-b287-2d08e4ef631c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:42.037910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:42.109230 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 175/340] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:59:42.123879 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7094a-a1e3-48ce-b26f-ed0805bf588f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:42.126821 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:42.171249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd4147-5cff-4919-9411-6ef5793f64b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:42.171910 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd4147-5cff-4919-9411-6ef5793f64b1 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:42.172193 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd4147-5cff-4919-9411-6ef5793f64b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:42.172459 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd4147-5cff-4919-9411-6ef5793f64b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:42.256854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd4147-5cff-4919-9411-6ef5793f64b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:42.258336 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd4147-5cff-4919-9411-6ef5793f64b1 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:42.372415 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:42.379401 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 166/341] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:42.385283 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0325746-37a4-4fcf-b287-2d08e4ef631c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:42.506312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0325746-37a4-4fcf-b287-2d08e4ef631c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:42.549142 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b99c7-b933-4252-a824-4306eaffe0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:42.549776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b99c7-b933-4252-a824-4306eaffe0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:42.550052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b99c7-b933-4252-a824-4306eaffe0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:42.550317 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b99c7-b933-4252-a824-4306eaffe0f0 None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:42.551492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b99c7-b933-4252-a824-4306eaffe0f0 None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:42.552993 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b99c7-b933-4252-a824-4306eaffe0f0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:42.555109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 176/342] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:42 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:42.563566 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd4147-5cff-4919-9411-6ef5793f64b1 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:42.677743 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd4147-5cff-4919-9411-6ef5793f64b1 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:42.681298 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e83eed-c055-437b-919a-82531d667b85 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:42.681611 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e83eed-c055-437b-919a-82531d667b85 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:42.681899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e83eed-c055-437b-919a-82531d667b85 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:42.683664 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 177/343] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:42.690428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbd917-6509-46f3-ac03-86b2108b8796 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:42.690847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbd917-6509-46f3-ac03-86b2108b8796 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:42.691082 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbd917-6509-46f3-ac03-86b2108b8796 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:42.692385 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 178/344] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:59:42.695944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de2c6a-a1ff-4311-910e-56582a992cb2 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:42.696223 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de2c6a-a1ff-4311-910e-56582a992cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:42.696495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de2c6a-a1ff-4311-910e-56582a992cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:42.719572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e2de2c6a-a1ff-4311-910e-56582a992cb2 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:42.721155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e2de2c6a-a1ff-4311-910e-56582a992cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:42.725523 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d8622-64b5-4bf2-a317-4fa65d9f9a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:42.726238 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d8622-64b5-4bf2-a317-4fa65d9f9a6d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:42.726541 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d8622-64b5-4bf2-a317-4fa65d9f9a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:42.726821 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d8622-64b5-4bf2-a317-4fa65d9f9a6d None admin] PATH_INFO: `/v3/groups/3f2aa215b75e44d2874532a46d371681` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:42.736811 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d8622-64b5-4bf2-a317-4fa65d9f9a6d None admin] RBAC: Authorizing `identity:get_group(group_id=3f2aa215b75e44d2874532a46d371681)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:42.737988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d8622-64b5-4bf2-a317-4fa65d9f9a6d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:42.740170 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 167/345] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 23 05:59:42 2022] GET /identity/v3/groups/3f2aa215b75e44d2874532a46d371681 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:42.745574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782dd9f-d205-4a6a-a8fb-313e183d7e5a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:42.745574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782dd9f-d205-4a6a-a8fb-313e183d7e5a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:42.745574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782dd9f-d205-4a6a-a8fb-313e183d7e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:42.762848 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:42.767384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1782dd9f-d205-4a6a-a8fb-313e183d7e5a None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:42.768866 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1782dd9f-d205-4a6a-a8fb-313e183d7e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:42.808642 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 179/346] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:59:42.809880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de2c6a-a1ff-4311-910e-56582a992cb2 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:42.840763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:42.858844 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 168/347] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:42.859868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782dd9f-d205-4a6a-a8fb-313e183d7e5a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:42.977476 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de2c6a-a1ff-4311-910e-56582a992cb2 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:43.022970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782dd9f-d205-4a6a-a8fb-313e183d7e5a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:43.040576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18779ccc-9cc8-4ab3-8d76-92714c50d4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:43.041321 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18779ccc-9cc8-4ab3-8d76-92714c50d4ac None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:43.041660 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18779ccc-9cc8-4ab3-8d76-92714c50d4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:43.041939 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18779ccc-9cc8-4ab3-8d76-92714c50d4ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:43.067497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18779ccc-9cc8-4ab3-8d76-92714c50d4ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-18779ccc-9cc8-4ab3-8d76-92714c50d4ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:43.072747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:43.073604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:43.074410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:43.074410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:43.074410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:43.074410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:43.074410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:43.074410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:43.074410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:43.074410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:43.074410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:43.074410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:43.074410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:43.074410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:43.074410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 180/348] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 05:59:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:43.081409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18779ccc-9cc8-4ab3-8d76-92714c50d4ac None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:43.084773 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ddb27-fd53-42ee-9339-a0fc27c285a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:43.085447 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ddb27-fd53-42ee-9339-a0fc27c285a4 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:43.085735 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ddb27-fd53-42ee-9339-a0fc27c285a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:43.086008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ddb27-fd53-42ee-9339-a0fc27c285a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:43.086780 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ddb27-fd53-42ee-9339-a0fc27c285a4 None admin] RBAC: Adding query filter params (group.id=3f2aa215b75e44d2874532a46d371681, role.id=a2f35dad17e94553b63d07f8b8d8f1a4, scope.project.id=bca76726c98f48f482c676027eb393d2) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:43.087084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ddb27-fd53-42ee-9339-a0fc27c285a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:43.088536 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ddb27-fd53-42ee-9339-a0fc27c285a4 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:43.096021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 169/349] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 23 05:59:42 2022] GET /identity/v3/role_assignments?group.id=3f2aa215b75e44d2874532a46d371681&scope.project.id=bca76726c98f48f482c676027eb393d2&role.id=a2f35dad17e94553b63d07f8b8d8f1a4 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:43.134459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:43.171429 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ddb27-fd53-42ee-9339-a0fc27c285a4 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:43.260344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:43.270824 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18779ccc-9cc8-4ab3-8d76-92714c50d4ac None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:43.312563 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0754e-0b51-46d7-8e00-1d84cbdbb15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:43.313420 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0754e-0b51-46d7-8e00-1d84cbdbb15c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:43.313844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0754e-0b51-46d7-8e00-1d84cbdbb15c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:43.314243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0754e-0b51-46d7-8e00-1d84cbdbb15c None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:43.315114 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0754e-0b51-46d7-8e00-1d84cbdbb15c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:43.315562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0754e-0b51-46d7-8e00-1d84cbdbb15c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:43.317222 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0754e-0b51-46d7-8e00-1d84cbdbb15c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:43.328741 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 181/350] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 05:59:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:43.328967 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ddb27-fd53-42ee-9339-a0fc27c285a4 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:43.334848 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0754e-0b51-46d7-8e00-1d84cbdbb15c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:43.372686 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c77bd-86ba-471b-972a-04509d234f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:43.373385 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c77bd-86ba-471b-972a-04509d234f67 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:43.373663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c77bd-86ba-471b-972a-04509d234f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:43.373935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c77bd-86ba-471b-972a-04509d234f67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:43.390894 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c77bd-86ba-471b-972a-04509d234f67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-e78c77bd-86ba-471b-972a-04509d234f67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:43.397900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:43.398987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:43.399834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:43.399834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:43.399834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:43.399834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:43.399834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:43.399834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:43.399834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:43.399834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:43.399834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:43.399834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:43.399834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:43.399834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:43.399834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 170/351] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:59:43.400603 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9d7fb-606d-478e-9792-2f1618fe154e None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:43.400881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9d7fb-606d-478e-9792-2f1618fe154e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:43.401156 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9d7fb-606d-478e-9792-2f1618fe154e None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:43.405134 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 171/352] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:43.405411 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9d7fb-606d-478e-9792-2f1618fe154e None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:43.457854 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:43.482035 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0754e-0b51-46d7-8e00-1d84cbdbb15c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:43.526388 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc7959-2582-4c15-8dc2-d47d0de58a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:43.527374 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc7959-2582-4c15-8dc2-d47d0de58a61 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:43.527897 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc7959-2582-4c15-8dc2-d47d0de58a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:43.528365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc7959-2582-4c15-8dc2-d47d0de58a61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:43.547525 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9d7fb-606d-478e-9792-2f1618fe154e None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:43.567828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:43.595681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1ae65-e7bf-4655-b2ba-f9c1fed0d06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:43.596319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1ae65-e7bf-4655-b2ba-f9c1fed0d06c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:43.597007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1ae65-e7bf-4655-b2ba-f9c1fed0d06c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:43.597288 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1ae65-e7bf-4655-b2ba-f9c1fed0d06c None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:43.598494 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1ae65-e7bf-4655-b2ba-f9c1fed0d06c None admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:43.598494 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1ae65-e7bf-4655-b2ba-f9c1fed0d06c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:43.600001 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1ae65-e7bf-4655-b2ba-f9c1fed0d06c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:43.616620 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 172/353] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:43.618484 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea43a8-d692-48a2-a39d-44e410356b7c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:43.618995 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea43a8-d692-48a2-a39d-44e410356b7c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:43.619436 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc7959-2582-4c15-8dc2-d47d0de58a61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:43.619715 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea43a8-d692-48a2-a39d-44e410356b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:43.620975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc7959-2582-4c15-8dc2-d47d0de58a61 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:43.643508 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-63ea43a8-d692-48a2-a39d-44e410356b7c None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:43.645195 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-63ea43a8-d692-48a2-a39d-44e410356b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:43.742275 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 173/354] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:43.743577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea43a8-d692-48a2-a39d-44e410356b7c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:43.757184 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:43.779792 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 182/355] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:43.780979 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc7959-2582-4c15-8dc2-d47d0de58a61 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:43.900359 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea43a8-d692-48a2-a39d-44e410356b7c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:43.914935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:43.958294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc7959-2582-4c15-8dc2-d47d0de58a61 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:43.963429 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b9e90-b97a-4ad3-87ea-d6b1d611c4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:43.964367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b9e90-b97a-4ad3-87ea-d6b1d611c4ab None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:43.964844 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b9e90-b97a-4ad3-87ea-d6b1d611c4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:43.965309 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b9e90-b97a-4ad3-87ea-d6b1d611c4ab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:43.988450 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b9e90-b97a-4ad3-87ea-d6b1d611c4ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-6a5b9e90-b97a-4ad3-87ea-d6b1d611c4ab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:43.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:43.995581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:43.996483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:43.996483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:43.996483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:43.996483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:43.996483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:43.996483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:43.996483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:43.996483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:43.996483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:43.996483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:43.996483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 05:59:43.996483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:43.996483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 174/356] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:43.997126 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b9e90-b97a-4ad3-87ea-d6b1d611c4ab None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:44.045810 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e83b65-b946-4635-86c6-ce57724c9e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:44.046337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e83b65-b946-4635-86c6-ce57724c9e24 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:44.046695 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e83b65-b946-4635-86c6-ce57724c9e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:44.046969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e83b65-b946-4635-86c6-ce57724c9e24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:44.124555 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b9e90-b97a-4ad3-87ea-d6b1d611c4ab None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:44.145064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:44.161181 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67635227-423b-4b4d-9134-f4b20d561da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:44.161822 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67635227-423b-4b4d-9134-f4b20d561da6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:44.162094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67635227-423b-4b4d-9134-f4b20d561da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:44.162359 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67635227-423b-4b4d-9134-f4b20d561da6 None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:44.163507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67635227-423b-4b4d-9134-f4b20d561da6 None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:44.164954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67635227-423b-4b4d-9134-f4b20d561da6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:44.166911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 175/357] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:43 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:44.167679 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67635227-423b-4b4d-9134-f4b20d561da6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:44.172007 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e83b65-b946-4635-86c6-ce57724c9e24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:44.173486 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e83b65-b946-4635-86c6-ce57724c9e24 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:44.205579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:44.280557 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67635227-423b-4b4d-9134-f4b20d561da6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:44.283266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 183/358] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:44.285276 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e83b65-b946-4635-86c6-ce57724c9e24 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:44.321642 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689aaba-48f2-48ff-bbf1-e35605523843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:44.322268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689aaba-48f2-48ff-bbf1-e35605523843 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:44.322808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689aaba-48f2-48ff-bbf1-e35605523843 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:44.322918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689aaba-48f2-48ff-bbf1-e35605523843 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:44.323717 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689aaba-48f2-48ff-bbf1-e35605523843 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:44.324123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689aaba-48f2-48ff-bbf1-e35605523843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:44.325633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689aaba-48f2-48ff-bbf1-e35605523843 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:44.339253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 176/359] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:44.339253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689aaba-48f2-48ff-bbf1-e35605523843 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:44.397801 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:44.416228 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e83b65-b946-4635-86c6-ce57724c9e24 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:44.454198 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:44.463369 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689aaba-48f2-48ff-bbf1-e35605523843 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:44.470401 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8284f68-e8a1-4b1b-b19d-b2d806f6b952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:44.471677 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8284f68-e8a1-4b1b-b19d-b2d806f6b952 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:44.472164 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8284f68-e8a1-4b1b-b19d-b2d806f6b952 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:44.472690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8284f68-e8a1-4b1b-b19d-b2d806f6b952 None admin] PATH_INFO: `/v3/groups/3f2aa215b75e44d2874532a46d371681` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:44.474853 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8284f68-e8a1-4b1b-b19d-b2d806f6b952 None admin] RBAC: Authorizing `identity:get_group(group_id=3f2aa215b75e44d2874532a46d371681)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:44.476308 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8284f68-e8a1-4b1b-b19d-b2d806f6b952 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:44.478065 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 184/360] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 23 05:59:44 2022] GET /identity/v3/groups/3f2aa215b75e44d2874532a46d371681 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:44.479085 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa371ae3-93be-4cb9-a67d-9b1368811500 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:44.479365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa371ae3-93be-4cb9-a67d-9b1368811500 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:44.479635 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa371ae3-93be-4cb9-a67d-9b1368811500 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:44.481004 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 185/361] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:44.481540 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa371ae3-93be-4cb9-a67d-9b1368811500 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:44.503436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84336b36-17ac-48d0-b245-b91661dec04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:44.504091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84336b36-17ac-48d0-b245-b91661dec04d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:44.504368 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84336b36-17ac-48d0-b245-b91661dec04d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:44.504638 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84336b36-17ac-48d0-b245-b91661dec04d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:44.519497 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84336b36-17ac-48d0-b245-b91661dec04d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-84336b36-17ac-48d0-b245-b91661dec04d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:44.525264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:44.526130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:44.526973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:44.526973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:44.526973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:44.526973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:44.526973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:44.526973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:44.526973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:44.526973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:44.526973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:44.526973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:44.526973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:44.526973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:44.526973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 177/362] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 05:59:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:44.527629 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84336b36-17ac-48d0-b245-b91661dec04d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:44.625783 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa371ae3-93be-4cb9-a67d-9b1368811500 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:44.660195 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84336b36-17ac-48d0-b245-b91661dec04d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:44.673595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ecc95-b32b-450b-9a0f-611341993079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:44.674370 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ecc95-b32b-450b-9a0f-611341993079 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:44.674696 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ecc95-b32b-450b-9a0f-611341993079 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:44.674961 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ecc95-b32b-450b-9a0f-611341993079 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:44.676072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ecc95-b32b-450b-9a0f-611341993079 None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:44.677522 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ecc95-b32b-450b-9a0f-611341993079 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:44.679223 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 186/363] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:44 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:44.679982 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ecc95-b32b-450b-9a0f-611341993079 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:44.709089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4a87b-ff03-4d6c-b758-b9d2be133a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:44.710102 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4a87b-ff03-4d6c-b758-b9d2be133a18 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:44.710382 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4a87b-ff03-4d6c-b758-b9d2be133a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:44.710699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4a87b-ff03-4d6c-b758-b9d2be133a18 None admin] PATH_INFO: `/v3/users/32bfde64b69e419db28c5daf0f7850ab` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:44.711445 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:44.713401 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4a87b-ff03-4d6c-b758-b9d2be133a18 None admin] RBAC: Authorizing `identity:get_user(user_id=32bfde64b69e419db28c5daf0f7850ab)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:44.714896 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4a87b-ff03-4d6c-b758-b9d2be133a18 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:44.721968 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 178/364] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 05:59:44 2022] GET /identity/v3/users/32bfde64b69e419db28c5daf0f7850ab => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:44.721968 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381e173-a58c-4a31-9c0a-2e0269adbca3 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:44.721968 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381e173-a58c-4a31-9c0a-2e0269adbca3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:44.721968 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381e173-a58c-4a31-9c0a-2e0269adbca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:44.745220 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-3381e173-a58c-4a31-9c0a-2e0269adbca3 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:44.748676 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3381e173-a58c-4a31-9c0a-2e0269adbca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:44.769733 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:44.822616 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ecc95-b32b-450b-9a0f-611341993079 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:44.834257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 179/365] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:44.835288 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381e173-a58c-4a31-9c0a-2e0269adbca3 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:44.864058 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cabfd-4452-4048-bfc3-c15b36ecce97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:44.864779 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cabfd-4452-4048-bfc3-c15b36ecce97 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:44.865056 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cabfd-4452-4048-bfc3-c15b36ecce97 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:44.865323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cabfd-4452-4048-bfc3-c15b36ecce97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:44.866052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cabfd-4452-4048-bfc3-c15b36ecce97 None admin] RBAC: Adding query filter params (group.id=3f2aa215b75e44d2874532a46d371681, role.id=f47755365f24477f9846225c9e91de2d, scope.project.id=bca76726c98f48f482c676027eb393d2) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:44.866353 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cabfd-4452-4048-bfc3-c15b36ecce97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:44.868264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cabfd-4452-4048-bfc3-c15b36ecce97 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:44.875515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 187/366] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 23 05:59:44 2022] GET /identity/v3/role_assignments?group.id=3f2aa215b75e44d2874532a46d371681&scope.project.id=bca76726c98f48f482c676027eb393d2&role.id=f47755365f24477f9846225c9e91de2d => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:44.876503 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cabfd-4452-4048-bfc3-c15b36ecce97 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:44.980453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381e173-a58c-4a31-9c0a-2e0269adbca3 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:45.001988 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:45.032633 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cabfd-4452-4048-bfc3-c15b36ecce97 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:45.046321 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa3d87-d807-44df-b040-7f991fdf6bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:45.047203 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa3d87-d807-44df-b040-7f991fdf6bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:45.047483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa3d87-d807-44df-b040-7f991fdf6bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:45.047755 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa3d87-d807-44df-b040-7f991fdf6bc6 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:45.048444 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa3d87-d807-44df-b040-7f991fdf6bc6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:45.048747 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa3d87-d807-44df-b040-7f991fdf6bc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:45.050406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa3d87-d807-44df-b040-7f991fdf6bc6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:45.069293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 180/367] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 05:59:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:45.072630 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa3d87-d807-44df-b040-7f991fdf6bc6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:45.109943 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:45.114993 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d313758-06df-4423-937d-1994d721745d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:45.115643 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d313758-06df-4423-937d-1994d721745d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:45.115927 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d313758-06df-4423-937d-1994d721745d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:45.116192 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d313758-06df-4423-937d-1994d721745d None admin] PATH_INFO: `/v3/groups/3f2aa215b75e44d2874532a46d371681` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:45.118628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d313758-06df-4423-937d-1994d721745d None admin] RBAC: Authorizing `identity:get_group(group_id=3f2aa215b75e44d2874532a46d371681)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:45.120527 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d313758-06df-4423-937d-1994d721745d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:45.123182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 188/368] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 23 05:59:44 2022] GET /identity/v3/groups/3f2aa215b75e44d2874532a46d371681 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:45.124068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d313758-06df-4423-937d-1994d721745d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:45.226390 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa3d87-d807-44df-b040-7f991fdf6bc6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:45.275345 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31e43a-0e33-400f-87a9-9e8fdf4ad6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:45.275976 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31e43a-0e33-400f-87a9-9e8fdf4ad6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:45.276254 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31e43a-0e33-400f-87a9-9e8fdf4ad6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:45.276523 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31e43a-0e33-400f-87a9-9e8fdf4ad6d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:45.277283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31e43a-0e33-400f-87a9-9e8fdf4ad6d9 None admin] RBAC: Adding query filter params (role.id=86e796f30478458d99f1d139c853a1db, user.id=32bfde64b69e419db28c5daf0f7850ab, scope.system=all) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:45.277568 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31e43a-0e33-400f-87a9-9e8fdf4ad6d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:45.279054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31e43a-0e33-400f-87a9-9e8fdf4ad6d9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:45.284370 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d313758-06df-4423-937d-1994d721745d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:45.296440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 181/369] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Aug 23 05:59:45 2022] GET /identity/v3/role_assignments?user.id=32bfde64b69e419db28c5daf0f7850ab&scope.system=all&role.id=86e796f30478458d99f1d139c853a1db => generated 510 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:45.297309 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31e43a-0e33-400f-87a9-9e8fdf4ad6d9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:45.348592 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb14b6-3ef3-426a-9bf6-8d3afee17caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:45.349618 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb14b6-3ef3-426a-9bf6-8d3afee17caa None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:45.350104 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb14b6-3ef3-426a-9bf6-8d3afee17caa None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:45.351123 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb14b6-3ef3-426a-9bf6-8d3afee17caa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:45.388442 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:45.416776 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:45.448506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31e43a-0e33-400f-87a9-9e8fdf4ad6d9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:45.488936 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bdbff-3a21-4e8c-b217-8ea5c2766d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:45.489907 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bdbff-3a21-4e8c-b217-8ea5c2766d65 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:45.490407 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bdbff-3a21-4e8c-b217-8ea5c2766d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:45.490901 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bdbff-3a21-4e8c-b217-8ea5c2766d65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:45.491853 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bdbff-3a21-4e8c-b217-8ea5c2766d65 None admin] RBAC: Adding query filter params (group.id=3f2aa215b75e44d2874532a46d371681, role.id=a2f35dad17e94553b63d07f8b8d8f1a4, scope.project.id=b4d0c8983a01433eb5c05fa508bede98) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:45.492370 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bdbff-3a21-4e8c-b217-8ea5c2766d65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:45.494381 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bdbff-3a21-4e8c-b217-8ea5c2766d65 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:45.501685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 182/370] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 23 05:59:45 2022] GET /identity/v3/role_assignments?group.id=3f2aa215b75e44d2874532a46d371681&scope.project.id=b4d0c8983a01433eb5c05fa508bede98&role.id=a2f35dad17e94553b63d07f8b8d8f1a4 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:45.503038 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bdbff-3a21-4e8c-b217-8ea5c2766d65 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:45.515795 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb14b6-3ef3-426a-9bf6-8d3afee17caa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:45.517358 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb14b6-3ef3-426a-9bf6-8d3afee17caa None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:45.694166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bdbff-3a21-4e8c-b217-8ea5c2766d65 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:45.696304 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 189/371] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:45.697599 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd50d5-7167-4f3c-a85e-cfca0ea75e33 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:45.697883 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd50d5-7167-4f3c-a85e-cfca0ea75e33 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:45.698155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd50d5-7167-4f3c-a85e-cfca0ea75e33 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:45.702375 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 190/372] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:45.702754 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd50d5-7167-4f3c-a85e-cfca0ea75e33 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:45.747780 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69f3eb-af7e-4de0-a938-74c4b3a9710b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:45.749801 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69f3eb-af7e-4de0-a938-74c4b3a9710b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:45.750100 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69f3eb-af7e-4de0-a938-74c4b3a9710b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:45.750363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69f3eb-af7e-4de0-a938-74c4b3a9710b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:45.758247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:45.771244 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69f3eb-af7e-4de0-a938-74c4b3a9710b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-fa69f3eb-af7e-4de0-a938-74c4b3a9710b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:45.777099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:45.778137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:45.779579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:45.779579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:45.779579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:45.779579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:45.779579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:45.779579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:45.779579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:45.779579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:45.779579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:45.779579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:45.779579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:45.779579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:45.779579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 183/373] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 05:59:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:45.780790 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a330b-7453-425f-8253-d33b94fdac5a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:45.781045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a330b-7453-425f-8253-d33b94fdac5a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:45.781325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a330b-7453-425f-8253-d33b94fdac5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:45.791802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:45.804717 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-776a330b-7453-425f-8253-d33b94fdac5a None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:45.806414 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-776a330b-7453-425f-8253-d33b94fdac5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:45.866355 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd50d5-7167-4f3c-a85e-cfca0ea75e33 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:45.894339 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 184/374] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:45.898539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a330b-7453-425f-8253-d33b94fdac5a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:45.927170 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375edfc-eae9-4452-943a-0fe8fb56686e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:45.927830 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375edfc-eae9-4452-943a-0fe8fb56686e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:45.928082 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375edfc-eae9-4452-943a-0fe8fb56686e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:45.928353 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375edfc-eae9-4452-943a-0fe8fb56686e None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:45.935203 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375edfc-eae9-4452-943a-0fe8fb56686e None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:45.936757 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375edfc-eae9-4452-943a-0fe8fb56686e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:45.938459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 191/375] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:45 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:45.944189 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375edfc-eae9-4452-943a-0fe8fb56686e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:46.063332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a330b-7453-425f-8253-d33b94fdac5a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:46.098921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:46.099716 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375edfc-eae9-4452-943a-0fe8fb56686e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:46.109525 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac693bcc-af6b-4474-a954-5c21448fd340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:46.110144 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac693bcc-af6b-4474-a954-5c21448fd340 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:46.110422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac693bcc-af6b-4474-a954-5c21448fd340 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:46.110724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac693bcc-af6b-4474-a954-5c21448fd340 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:46.111399 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac693bcc-af6b-4474-a954-5c21448fd340 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:46.111701 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac693bcc-af6b-4474-a954-5c21448fd340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:46.114283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac693bcc-af6b-4474-a954-5c21448fd340 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:46.126295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 185/376] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:46.129196 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:46.132826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac693bcc-af6b-4474-a954-5c21448fd340 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:46.143711 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d7a1d-727c-4e89-a048-fac690d1c74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:46.144343 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d7a1d-727c-4e89-a048-fac690d1c74b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:46.145218 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d7a1d-727c-4e89-a048-fac690d1c74b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:46.145494 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d7a1d-727c-4e89-a048-fac690d1c74b None admin] PATH_INFO: `/v3/groups/3f2aa215b75e44d2874532a46d371681` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:46.146654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d7a1d-727c-4e89-a048-fac690d1c74b None admin] RBAC: Authorizing `identity:get_group(group_id=3f2aa215b75e44d2874532a46d371681)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:46.148970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d7a1d-727c-4e89-a048-fac690d1c74b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:46.151461 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 192/377] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 23 05:59:45 2022] GET /identity/v3/groups/3f2aa215b75e44d2874532a46d371681 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:46.158703 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d7a1d-727c-4e89-a048-fac690d1c74b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:46.272281 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac693bcc-af6b-4474-a954-5c21448fd340 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:46.295586 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d7a1d-727c-4e89-a048-fac690d1c74b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:46.314829 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a774644-3daa-4964-946f-0dfec3c68e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:46.315430 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a774644-3daa-4964-946f-0dfec3c68e81 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:46.315817 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a774644-3daa-4964-946f-0dfec3c68e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:46.316086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a774644-3daa-4964-946f-0dfec3c68e81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:46.346121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52de494-9b65-4ff8-b394-dbc6d8ca47dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:46.347078 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52de494-9b65-4ff8-b394-dbc6d8ca47dc None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:46.347356 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52de494-9b65-4ff8-b394-dbc6d8ca47dc None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:46.347858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52de494-9b65-4ff8-b394-dbc6d8ca47dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:46.349392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52de494-9b65-4ff8-b394-dbc6d8ca47dc None admin] RBAC: Adding query filter params (group.id=3f2aa215b75e44d2874532a46d371681, role.id=f47755365f24477f9846225c9e91de2d, scope.project.id=b4d0c8983a01433eb5c05fa508bede98) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:46.349956 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52de494-9b65-4ff8-b394-dbc6d8ca47dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:46.351442 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52de494-9b65-4ff8-b394-dbc6d8ca47dc None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:46.360184 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 193/378] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 23 05:59:46 2022] GET /identity/v3/role_assignments?group.id=3f2aa215b75e44d2874532a46d371681&scope.project.id=b4d0c8983a01433eb5c05fa508bede98&role.id=f47755365f24477f9846225c9e91de2d => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:46.361401 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52de494-9b65-4ff8-b394-dbc6d8ca47dc None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:46.393687 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:46.409212 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a774644-3daa-4964-946f-0dfec3c68e81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:46.410837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a774644-3daa-4964-946f-0dfec3c68e81 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:46.437684 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:46.527437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 186/379] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:46.536476 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a774644-3daa-4964-946f-0dfec3c68e81 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:46.536657 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52de494-9b65-4ff8-b394-dbc6d8ca47dc None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:46.615494 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce1301-02ea-4b7e-bd16-d57d978b09a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:46.616022 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce1301-02ea-4b7e-bd16-d57d978b09a5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:46.616413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce1301-02ea-4b7e-bd16-d57d978b09a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:46.616678 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce1301-02ea-4b7e-bd16-d57d978b09a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:46.641800 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce1301-02ea-4b7e-bd16-d57d978b09a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-87ce1301-02ea-4b7e-bd16-d57d978b09a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:46.648496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:46.649378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:46.650158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:46.650158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:46.650158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:46.650158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:46.650158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:46.650158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:46.650158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:46.650158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:46.650158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:46.650158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:46.650158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:46.650158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:46.650158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 194/380] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:46.651970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:46.659268 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce1301-02ea-4b7e-bd16-d57d978b09a5 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:46.672300 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a774644-3daa-4964-946f-0dfec3c68e81 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:46.751290 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a5831-aa8b-492d-9e34-31c3bc4bea53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:46.751938 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a5831-aa8b-492d-9e34-31c3bc4bea53 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:46.752264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a5831-aa8b-492d-9e34-31c3bc4bea53 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:46.752539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a5831-aa8b-492d-9e34-31c3bc4bea53 None admin] PATH_INFO: `/v3/projects/a1e3a96fd05742d9a58f3a78e19ef4f6` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:46.753772 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a5831-aa8b-492d-9e34-31c3bc4bea53 None admin] RBAC: Authorizing `identity:get_project(project_id=a1e3a96fd05742d9a58f3a78e19ef4f6)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:46.755272 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a5831-aa8b-492d-9e34-31c3bc4bea53 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:46.757403 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 187/381] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:46 2022] GET /identity/v3/projects/a1e3a96fd05742d9a58f3a78e19ef4f6 => generated 344 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:46.762791 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a5831-aa8b-492d-9e34-31c3bc4bea53 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:46.865455 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce1301-02ea-4b7e-bd16-d57d978b09a5 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:46.886926 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:46.943253 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9135d-044e-4d84-8532-49cc1425522a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:46.943882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9135d-044e-4d84-8532-49cc1425522a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:46.944160 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9135d-044e-4d84-8532-49cc1425522a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:46.944433 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9135d-044e-4d84-8532-49cc1425522a None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:46.945112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9135d-044e-4d84-8532-49cc1425522a None admin] RBAC: Adding query filter params (name=member) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:46.945416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9135d-044e-4d84-8532-49cc1425522a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:46.946833 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9135d-044e-4d84-8532-49cc1425522a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:46.947791 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a5831-aa8b-492d-9e34-31c3bc4bea53 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:46.972859 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 195/382] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:46.984022 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9135d-044e-4d84-8532-49cc1425522a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:47.015204 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc9dd6-161f-4522-97fb-126195aa6606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:47.015852 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc9dd6-161f-4522-97fb-126195aa6606 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:47.016129 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc9dd6-161f-4522-97fb-126195aa6606 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:47.016400 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc9dd6-161f-4522-97fb-126195aa6606 None admin] PATH_INFO: `/v3/groups/96ee92e558184c919a319c47bfc5809d` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:47.029019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc9dd6-161f-4522-97fb-126195aa6606 None admin] RBAC: Authorizing `identity:get_group(group_id=96ee92e558184c919a319c47bfc5809d)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:47.032335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc9dd6-161f-4522-97fb-126195aa6606 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:47.034063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 188/383] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 23 05:59:46 2022] GET /identity/v3/groups/96ee92e558184c919a319c47bfc5809d => generated 229 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:47.048889 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc9dd6-161f-4522-97fb-126195aa6606 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:47.084698 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:47.186164 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9135d-044e-4d84-8532-49cc1425522a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:47.205982 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc9dd6-161f-4522-97fb-126195aa6606 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:47.253083 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb14da-0faf-44e5-bb60-644efdabcfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:47.253783 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb14da-0faf-44e5-bb60-644efdabcfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:47.254066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb14da-0faf-44e5-bb60-644efdabcfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:47.254342 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb14da-0faf-44e5-bb60-644efdabcfc6 None admin] PATH_INFO: `/v3/groups/3f2aa215b75e44d2874532a46d371681` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:47.256983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb14da-0faf-44e5-bb60-644efdabcfc6 None admin] RBAC: Authorizing `identity:get_group(group_id=3f2aa215b75e44d2874532a46d371681)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:47.258424 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb14da-0faf-44e5-bb60-644efdabcfc6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:47.268731 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 196/384] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 23 05:59:46 2022] GET /identity/v3/groups/3f2aa215b75e44d2874532a46d371681 => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:47.272641 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb14da-0faf-44e5-bb60-644efdabcfc6 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:47.284128 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0a079-e20f-4e06-ae82-3ae33d64ae6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:47.284742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0a079-e20f-4e06-ae82-3ae33d64ae6d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:47.285023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0a079-e20f-4e06-ae82-3ae33d64ae6d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:47.285293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0a079-e20f-4e06-ae82-3ae33d64ae6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:47.286028 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0a079-e20f-4e06-ae82-3ae33d64ae6d None admin] RBAC: Adding query filter params (group.id=96ee92e558184c919a319c47bfc5809d, role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=a1e3a96fd05742d9a58f3a78e19ef4f6) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:47.286335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0a079-e20f-4e06-ae82-3ae33d64ae6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:47.288144 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0a079-e20f-4e06-ae82-3ae33d64ae6d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:47.302722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 189/385] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 23 05:59:47 2022] GET /identity/v3/role_assignments?group.id=96ee92e558184c919a319c47bfc5809d&scope.project.id=a1e3a96fd05742d9a58f3a78e19ef4f6&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 260 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:47.366955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:47.461883 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb14da-0faf-44e5-bb60-644efdabcfc6 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:47.484466 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3110ec-07d2-43d5-8eba-574cb6d3fc62 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:47.484770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3110ec-07d2-43d5-8eba-574cb6d3fc62 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:47.485005 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3110ec-07d2-43d5-8eba-574cb6d3fc62 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:47.486653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 190/386] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:47.491831 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c231e-5aaa-4c96-b55a-ffeb155bf725 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:47.492107 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c231e-5aaa-4c96-b55a-ffeb155bf725 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:47.492378 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c231e-5aaa-4c96-b55a-ffeb155bf725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:47.519040 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233d4d9-d241-4f87-820f-31811048e445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:47.519862 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233d4d9-d241-4f87-820f-31811048e445 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:47.520143 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233d4d9-d241-4f87-820f-31811048e445 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:47.520413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233d4d9-d241-4f87-820f-31811048e445 None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:47.522971 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233d4d9-d241-4f87-820f-31811048e445 None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:47.524435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233d4d9-d241-4f87-820f-31811048e445 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:47.526102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 197/387] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:47 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:47.530820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-799c231e-5aaa-4c96-b55a-ffeb155bf725 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:47.532019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-799c231e-5aaa-4c96-b55a-ffeb155bf725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:47.533495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233d4d9-d241-4f87-820f-31811048e445 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:47.651738 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233d4d9-d241-4f87-820f-31811048e445 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:47.667257 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:47.692483 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e8b4a-da3a-4ac6-9ce8-e2eb5a0fcccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:47.693145 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 191/388] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:47.693254 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e8b4a-da3a-4ac6-9ce8-e2eb5a0fcccb None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:47.693507 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e8b4a-da3a-4ac6-9ce8-e2eb5a0fcccb None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:47.693787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e8b4a-da3a-4ac6-9ce8-e2eb5a0fcccb None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2/groups/3f2aa215b75e44d2874532a46d371681/roles/a2f35dad17e94553b63d07f8b8d8f1a4` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:47.696010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e8b4a-da3a-4ac6-9ce8-e2eb5a0fcccb None admin] RBAC: Authorizing `identity:create_grant(project_id=bca76726c98f48f482c676027eb393d2, group_id=3f2aa215b75e44d2874532a46d371681, role_id=a2f35dad17e94553b63d07f8b8d8f1a4)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:47.697472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e8b4a-da3a-4ac6-9ce8-e2eb5a0fcccb None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:47.718560 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 198/389] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Aug 23 05:59:47 2022] PUT /identity/v3/projects/bca76726c98f48f482c676027eb393d2/groups/3f2aa215b75e44d2874532a46d371681/roles/a2f35dad17e94553b63d07f8b8d8f1a4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 05:59:48.098603 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c231e-5aaa-4c96-b55a-ffeb155bf725 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:48.122412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:48.212384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c231e-5aaa-4c96-b55a-ffeb155bf725 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:48.251367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c834f-a0cd-4ec5-91fc-7e519087482d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:48.252005 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c834f-a0cd-4ec5-91fc-7e519087482d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:48.252284 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c834f-a0cd-4ec5-91fc-7e519087482d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:48.252558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c834f-a0cd-4ec5-91fc-7e519087482d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:48.268456 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c834f-a0cd-4ec5-91fc-7e519087482d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-4b4c834f-a0cd-4ec5-91fc-7e519087482d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:48.273274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:48.274175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:48.275062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:48.275062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:48.275062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:48.275062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:48.275062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:48.275062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:48.275062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:48.275062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:48.275062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:48.275062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:48.275062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:48.275062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:48.275062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 192/390] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 05:59:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:48.281856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e8b4a-da3a-4ac6-9ce8-e2eb5a0fcccb None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:48.406102 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6fc54-49d3-4c05-a40a-8e90ad65ac3d None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:48.406772 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6fc54-49d3-4c05-a40a-8e90ad65ac3d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:48.406772 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6fc54-49d3-4c05-a40a-8e90ad65ac3d None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:48.407619 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e8b4a-da3a-4ac6-9ce8-e2eb5a0fcccb None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:48.408114 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 193/391] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:48.413202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bce95-8b0f-49c9-ab93-3d6f6bbe97e9 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:48.413467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bce95-8b0f-49c9-ab93-3d6f6bbe97e9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:48.413739 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bce95-8b0f-49c9-ab93-3d6f6bbe97e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:48.441407 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-493bce95-8b0f-49c9-ab93-3d6f6bbe97e9 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:48.444262 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-493bce95-8b0f-49c9-ab93-3d6f6bbe97e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:48.481227 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a806ee-2ca3-44f7-b2fe-ddbfb9e21a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:48.481867 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a806ee-2ca3-44f7-b2fe-ddbfb9e21a43 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:48.482152 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a806ee-2ca3-44f7-b2fe-ddbfb9e21a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:48.482501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a806ee-2ca3-44f7-b2fe-ddbfb9e21a43 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:48.483211 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a806ee-2ca3-44f7-b2fe-ddbfb9e21a43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:48.483518 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a806ee-2ca3-44f7-b2fe-ddbfb9e21a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:48.484905 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a806ee-2ca3-44f7-b2fe-ddbfb9e21a43 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:48.509204 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 199/392] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 05:59:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:48.519528 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a806ee-2ca3-44f7-b2fe-ddbfb9e21a43 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:48.598655 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 194/393] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:48.601188 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:48.697689 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a806ee-2ca3-44f7-b2fe-ddbfb9e21a43 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:48.736498 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232ba66-5df1-4f84-88bf-3b562a1f77a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:48.737205 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232ba66-5df1-4f84-88bf-3b562a1f77a0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:48.737707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232ba66-5df1-4f84-88bf-3b562a1f77a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:48.737707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232ba66-5df1-4f84-88bf-3b562a1f77a0 None admin] PATH_INFO: `/v3/groups/3f2aa215b75e44d2874532a46d371681` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:48.738886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232ba66-5df1-4f84-88bf-3b562a1f77a0 None admin] RBAC: Authorizing `identity:get_group(group_id=3f2aa215b75e44d2874532a46d371681)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:48.740347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232ba66-5df1-4f84-88bf-3b562a1f77a0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:48.742266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 200/394] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 23 05:59:48 2022] GET /identity/v3/groups/3f2aa215b75e44d2874532a46d371681 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:48.751264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bce95-8b0f-49c9-ab93-3d6f6bbe97e9 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:48.762380 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:48.872734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bce95-8b0f-49c9-ab93-3d6f6bbe97e9 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:48.916394 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed24b30-34da-479b-888a-aa095c15644a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:48.917089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed24b30-34da-479b-888a-aa095c15644a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:48.917373 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed24b30-34da-479b-888a-aa095c15644a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:48.917648 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed24b30-34da-479b-888a-aa095c15644a None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:48.919633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed24b30-34da-479b-888a-aa095c15644a None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:48.921591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed24b30-34da-479b-888a-aa095c15644a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:48.924384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 195/395] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:48 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:48.930077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232ba66-5df1-4f84-88bf-3b562a1f77a0 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:49.058062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed24b30-34da-479b-888a-aa095c15644a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:49.082223 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232ba66-5df1-4f84-88bf-3b562a1f77a0 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:49.083954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:49.120944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12a3b8-36ea-4d0b-b75f-2fb31d18d971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:49.121804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12a3b8-36ea-4d0b-b75f-2fb31d18d971 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:49.122089 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12a3b8-36ea-4d0b-b75f-2fb31d18d971 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:49.122366 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12a3b8-36ea-4d0b-b75f-2fb31d18d971 None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2/groups/3f2aa215b75e44d2874532a46d371681/roles/f47755365f24477f9846225c9e91de2d` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:49.134543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12a3b8-36ea-4d0b-b75f-2fb31d18d971 None admin] RBAC: Authorizing `identity:create_grant(project_id=bca76726c98f48f482c676027eb393d2, group_id=3f2aa215b75e44d2874532a46d371681, role_id=f47755365f24477f9846225c9e91de2d)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:49.134543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12a3b8-36ea-4d0b-b75f-2fb31d18d971 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:49.154412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 201/396] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Aug 23 05:59:48 2022] PUT /identity/v3/projects/bca76726c98f48f482c676027eb393d2/groups/3f2aa215b75e44d2874532a46d371681/roles/f47755365f24477f9846225c9e91de2d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 05:59:49.158557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375a2c0-73eb-453a-abf2-79ee1127de19 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:49.158557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375a2c0-73eb-453a-abf2-79ee1127de19 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:49.158557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375a2c0-73eb-453a-abf2-79ee1127de19 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:49.158973 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 202/397] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:49.170542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a31487-be69-4bb0-b6fe-70d7879c9eeb None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:49.170542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a31487-be69-4bb0-b6fe-70d7879c9eeb None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:49.170542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a31487-be69-4bb0-b6fe-70d7879c9eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:49.233302 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-57a31487-be69-4bb0-b6fe-70d7879c9eeb None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:49.234733 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-57a31487-be69-4bb0-b6fe-70d7879c9eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:49.256176 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:49.287623 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed24b30-34da-479b-888a-aa095c15644a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:49.325688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cdff16-aa64-4e5c-843c-29a16c71c9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:49.326831 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cdff16-aa64-4e5c-843c-29a16c71c9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:49.327338 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cdff16-aa64-4e5c-843c-29a16c71c9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:49.327796 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cdff16-aa64-4e5c-843c-29a16c71c9e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:49.339285 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cdff16-aa64-4e5c-843c-29a16c71c9e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-41cdff16-aa64-4e5c-843c-29a16c71c9e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:49.345695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:49.346681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:49.347440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:49.347440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:49.347440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:49.347440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:49.347440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:49.347440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:49.347440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:49.347440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:49.347440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:49.347440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:49.347440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:49.347440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:49.349680 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 196/398] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:59:49.358320 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cdff16-aa64-4e5c-843c-29a16c71c9e5 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:49.389781 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 203/399] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:49.513288 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cdff16-aa64-4e5c-843c-29a16c71c9e5 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:49.551261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457f2e5-325a-47f4-b6c3-a7ee13202104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:49.551886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457f2e5-325a-47f4-b6c3-a7ee13202104 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:49.552169 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457f2e5-325a-47f4-b6c3-a7ee13202104 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:49.552442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457f2e5-325a-47f4-b6c3-a7ee13202104 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:49.553244 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457f2e5-325a-47f4-b6c3-a7ee13202104 None admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:49.553552 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457f2e5-325a-47f4-b6c3-a7ee13202104 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:49.555023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457f2e5-325a-47f4-b6c3-a7ee13202104 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:49.556704 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:49.567651 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 197/400] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:49.573367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a31487-be69-4bb0-b6fe-70d7879c9eeb None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:49.702778 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:49.713035 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a31487-be69-4bb0-b6fe-70d7879c9eeb None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:49.752926 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0a6c2-5f6f-497e-80e5-4b0c76d00554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:49.753557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0a6c2-5f6f-497e-80e5-4b0c76d00554 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:49.753836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0a6c2-5f6f-497e-80e5-4b0c76d00554 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:49.754111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0a6c2-5f6f-497e-80e5-4b0c76d00554 None admin] PATH_INFO: `/v3/groups/3f2aa215b75e44d2874532a46d371681` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:49.755381 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0a6c2-5f6f-497e-80e5-4b0c76d00554 None admin] RBAC: Authorizing `identity:get_group(group_id=3f2aa215b75e44d2874532a46d371681)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:49.756829 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0a6c2-5f6f-497e-80e5-4b0c76d00554 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:49.762873 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 204/401] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 23 05:59:49 2022] GET /identity/v3/groups/3f2aa215b75e44d2874532a46d371681 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:49.765076 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457f2e5-325a-47f4-b6c3-a7ee13202104 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:49.874257 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0a6c2-5f6f-497e-80e5-4b0c76d00554 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:49.930866 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457f2e5-325a-47f4-b6c3-a7ee13202104 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:49.981391 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6020ee1-4125-48bf-b505-4cfecd33ca24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:49.983098 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6020ee1-4125-48bf-b505-4cfecd33ca24 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:49.983907 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6020ee1-4125-48bf-b505-4cfecd33ca24 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:49.984412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6020ee1-4125-48bf-b505-4cfecd33ca24 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:49.985884 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6020ee1-4125-48bf-b505-4cfecd33ca24 None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:49.988066 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6020ee1-4125-48bf-b505-4cfecd33ca24 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:49.989922 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 198/402] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:49 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:49.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daa61c-32a0-40ed-93cb-f638bf8e9e06 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:49.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daa61c-32a0-40ed-93cb-f638bf8e9e06 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:49.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daa61c-32a0-40ed-93cb-f638bf8e9e06 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:49.999937 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 199/403] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:50.001111 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9daa61c-32a0-40ed-93cb-f638bf8e9e06 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:50.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0a6c2-5f6f-497e-80e5-4b0c76d00554 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:50.063593 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a65aa3-9068-420b-8a4b-a7426bc9db82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:50.064322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a65aa3-9068-420b-8a4b-a7426bc9db82 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:50.064791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a65aa3-9068-420b-8a4b-a7426bc9db82 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:50.064921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a65aa3-9068-420b-8a4b-a7426bc9db82 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:50.076198 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a65aa3-9068-420b-8a4b-a7426bc9db82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-38a65aa3-9068-420b-8a4b-a7426bc9db82 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:50.081347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:50.082243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:50.083074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:50.083074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:50.083074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:50.083074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:50.083074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:50.083074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:50.083074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:50.083074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:50.083074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:50.083074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:50.083074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:50.083074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:50.086948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 205/404] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 05:59:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:50.088299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c935354-ff19-43d3-b460-fe7879a55ffa None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:50.088659 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c935354-ff19-43d3-b460-fe7879a55ffa None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:50.089088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c935354-ff19-43d3-b460-fe7879a55ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:50.092351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:50.111950 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-7c935354-ff19-43d3-b460-fe7879a55ffa None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:50.113153 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7c935354-ff19-43d3-b460-fe7879a55ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:50.131658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:50.149920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9daa61c-32a0-40ed-93cb-f638bf8e9e06 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:50.196872 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 206/405] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:50.199042 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c935354-ff19-43d3-b460-fe7879a55ffa None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:50.205140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd01ddc-9608-47d9-92e1-a1bd70a63d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:50.206192 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd01ddc-9608-47d9-92e1-a1bd70a63d62 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:50.206676 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd01ddc-9608-47d9-92e1-a1bd70a63d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:50.207173 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd01ddc-9608-47d9-92e1-a1bd70a63d62 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98/groups/3f2aa215b75e44d2874532a46d371681/roles/a2f35dad17e94553b63d07f8b8d8f1a4` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:50.210494 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd01ddc-9608-47d9-92e1-a1bd70a63d62 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4d0c8983a01433eb5c05fa508bede98, group_id=3f2aa215b75e44d2874532a46d371681, role_id=a2f35dad17e94553b63d07f8b8d8f1a4)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:50.215876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd01ddc-9608-47d9-92e1-a1bd70a63d62 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:50.243839 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 200/406] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Aug 23 05:59:50 2022] PUT /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98/groups/3f2aa215b75e44d2874532a46d371681/roles/a2f35dad17e94553b63d07f8b8d8f1a4 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 05:59:50.375365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c935354-ff19-43d3-b460-fe7879a55ffa None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:50.425969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f9118-2f21-4e52-b23d-07f2c9a0bbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:50.426634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f9118-2f21-4e52-b23d-07f2c9a0bbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:50.426932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f9118-2f21-4e52-b23d-07f2c9a0bbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:50.427194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f9118-2f21-4e52-b23d-07f2c9a0bbf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:50.428093 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f9118-2f21-4e52-b23d-07f2c9a0bbf9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:50.428394 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f9118-2f21-4e52-b23d-07f2c9a0bbf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:50.429816 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f9118-2f21-4e52-b23d-07f2c9a0bbf9 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:50.442387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 207/407] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 05:59:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:50.448069 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd01ddc-9608-47d9-92e1-a1bd70a63d62 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:50.471918 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9943a-8c34-4531-a8df-01eff1a81ba1 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:50.472240 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9943a-8c34-4531-a8df-01eff1a81ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:50.472455 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9943a-8c34-4531-a8df-01eff1a81ba1 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:50.473919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 208/408] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 05:59:50.479763 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad4f18-ff20-4805-b7d4-6fee7d020f04 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:50.480041 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad4f18-ff20-4805-b7d4-6fee7d020f04 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:50.480310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad4f18-ff20-4805-b7d4-6fee7d020f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:50.504308 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-6bad4f18-ff20-4805-b7d4-6fee7d020f04 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:50.505119 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6bad4f18-ff20-4805-b7d4-6fee7d020f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:50.512990 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:50.579111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 209/409] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:59:50.584387 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd01ddc-9608-47d9-92e1-a1bd70a63d62 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:50.603531 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad4f18-ff20-4805-b7d4-6fee7d020f04 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:50.611432 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:50.642289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf331fd-ec42-40ac-817b-1b9bb87a0cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:50.643053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf331fd-ec42-40ac-817b-1b9bb87a0cea None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:50.643433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf331fd-ec42-40ac-817b-1b9bb87a0cea None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:50.643805 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf331fd-ec42-40ac-817b-1b9bb87a0cea None admin] PATH_INFO: `/v3/groups/3f2aa215b75e44d2874532a46d371681` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:50.645518 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf331fd-ec42-40ac-817b-1b9bb87a0cea None admin] RBAC: Authorizing `identity:get_group(group_id=3f2aa215b75e44d2874532a46d371681)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:50.647329 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf331fd-ec42-40ac-817b-1b9bb87a0cea None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:50.649067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 201/410] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 23 05:59:50 2022] GET /identity/v3/groups/3f2aa215b75e44d2874532a46d371681 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:50.659642 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf331fd-ec42-40ac-817b-1b9bb87a0cea None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:50.783012 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad4f18-ff20-4805-b7d4-6fee7d020f04 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:50.799606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf331fd-ec42-40ac-817b-1b9bb87a0cea None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:50.850419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd24815-9777-4d78-8de4-d7ecfa0363c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:50.851569 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd24815-9777-4d78-8de4-d7ecfa0363c7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:50.852279 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd24815-9777-4d78-8de4-d7ecfa0363c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:50.852849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd24815-9777-4d78-8de4-d7ecfa0363c7 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:50.854838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd24815-9777-4d78-8de4-d7ecfa0363c7 None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:50.858592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd24815-9777-4d78-8de4-d7ecfa0363c7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:50.861128 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 202/411] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:50 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:50.866977 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd24815-9777-4d78-8de4-d7ecfa0363c7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:50.880878 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a151b-d616-49b9-a795-eb4cb9cdaec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:50.881499 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a151b-d616-49b9-a795-eb4cb9cdaec2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:50.881779 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a151b-d616-49b9-a795-eb4cb9cdaec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:50.882044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a151b-d616-49b9-a795-eb4cb9cdaec2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:50.918830 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a151b-d616-49b9-a795-eb4cb9cdaec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-1e2a151b-d616-49b9-a795-eb4cb9cdaec2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:50.924255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:50.925125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:50.925902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:50.925902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:50.925902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:50.925902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:50.925902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:50.925902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:50.925902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:50.925902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:50.925902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:50.925902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:50.925902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:50.925902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:50.931119 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 210/412] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 05:59:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:59:50.943491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:50.943694 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a151b-d616-49b9-a795-eb4cb9cdaec2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:50.961926 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:50.992810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd24815-9777-4d78-8de4-d7ecfa0363c7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:51.030916 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c16321-6288-4f5d-9886-fcf5a6c61d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:51.031717 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c16321-6288-4f5d-9886-fcf5a6c61d87 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:51.032713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c16321-6288-4f5d-9886-fcf5a6c61d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:51.032713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c16321-6288-4f5d-9886-fcf5a6c61d87 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98/groups/3f2aa215b75e44d2874532a46d371681/roles/f47755365f24477f9846225c9e91de2d` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:51.034731 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c16321-6288-4f5d-9886-fcf5a6c61d87 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4d0c8983a01433eb5c05fa508bede98, group_id=3f2aa215b75e44d2874532a46d371681, role_id=f47755365f24477f9846225c9e91de2d)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:51.036167 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c16321-6288-4f5d-9886-fcf5a6c61d87 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:51.053464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 203/413] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Aug 23 05:59:50 2022] PUT /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98/groups/3f2aa215b75e44d2874532a46d371681/roles/f47755365f24477f9846225c9e91de2d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 05:59:51.054305 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c16321-6288-4f5d-9886-fcf5a6c61d87 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:51.065022 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a151b-d616-49b9-a795-eb4cb9cdaec2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:51.124312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051ee4a-8641-4c00-bf69-d62d14f81c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:51.125244 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051ee4a-8641-4c00-bf69-d62d14f81c01 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:51.125724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051ee4a-8641-4c00-bf69-d62d14f81c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:51.126161 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051ee4a-8641-4c00-bf69-d62d14f81c01 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:51.127068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051ee4a-8641-4c00-bf69-d62d14f81c01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:51.127786 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051ee4a-8641-4c00-bf69-d62d14f81c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:51.130848 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051ee4a-8641-4c00-bf69-d62d14f81c01 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:51.147918 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 211/414] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:51.158541 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051ee4a-8641-4c00-bf69-d62d14f81c01 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:51.277873 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c16321-6288-4f5d-9886-fcf5a6c61d87 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:51.281144 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:51.311190 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:51.330175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a28e69-1094-4719-8181-5970a5f57a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:51.331095 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a28e69-1094-4719-8181-5970a5f57a86 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:51.331391 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a28e69-1094-4719-8181-5970a5f57a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:51.331659 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a28e69-1094-4719-8181-5970a5f57a86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:51.336145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051ee4a-8641-4c00-bf69-d62d14f81c01 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:51.344473 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a28e69-1094-4719-8181-5970a5f57a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-14a28e69-1094-4719-8181-5970a5f57a86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:51.351584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:51.352449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:51.353239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:51.353239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:51.353239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:51.353239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:51.353239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:51.353239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:51.353239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:51.353239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:51.353239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:51.353239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:51.353239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:51.353239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:51.353239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 204/415] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:51.359627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a28e69-1094-4719-8181-5970a5f57a86 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:51.396655 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea351cdc-deff-4bb1-91e5-11f969ac4ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:51.397302 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea351cdc-deff-4bb1-91e5-11f969ac4ece None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:51.397919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea351cdc-deff-4bb1-91e5-11f969ac4ece None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:51.397919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea351cdc-deff-4bb1-91e5-11f969ac4ece None admin] PATH_INFO: `/v3/groups/96ee92e558184c919a319c47bfc5809d` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:51.399138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea351cdc-deff-4bb1-91e5-11f969ac4ece None admin] RBAC: Authorizing `identity:get_group(group_id=96ee92e558184c919a319c47bfc5809d)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:51.400682 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea351cdc-deff-4bb1-91e5-11f969ac4ece None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:51.402768 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 212/416] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 23 05:59:51 2022] GET /identity/v3/groups/96ee92e558184c919a319c47bfc5809d => generated 229 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:51.410294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea351cdc-deff-4bb1-91e5-11f969ac4ece None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:51.480039 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a28e69-1094-4719-8181-5970a5f57a86 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:51.517389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2038a-78c2-42d6-98fa-a00ca8e79593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:51.517999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2038a-78c2-42d6-98fa-a00ca8e79593 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:51.518278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2038a-78c2-42d6-98fa-a00ca8e79593 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:51.518718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2038a-78c2-42d6-98fa-a00ca8e79593 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:51.519415 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2038a-78c2-42d6-98fa-a00ca8e79593 None admin] RBAC: Adding query filter params (name=member) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:51.519714 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2038a-78c2-42d6-98fa-a00ca8e79593 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:51.521094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2038a-78c2-42d6-98fa-a00ca8e79593 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:51.534686 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 205/417] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:51.534686 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f22b48-28c9-4f8c-99de-c2324b857bbc None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:51.534686 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f22b48-28c9-4f8c-99de-c2324b857bbc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:51.534686 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f22b48-28c9-4f8c-99de-c2324b857bbc None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:51.534686 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 206/418] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:51.539801 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f22b48-28c9-4f8c-99de-c2324b857bbc None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:51.546285 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea351cdc-deff-4bb1-91e5-11f969ac4ece None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:51.584017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:51.588153 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:51.592584 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e58ee-0e03-4072-a4a1-d581b3b0d798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:51.593809 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e58ee-0e03-4072-a4a1-d581b3b0d798 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:51.594158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e58ee-0e03-4072-a4a1-d581b3b0d798 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:51.594493 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e58ee-0e03-4072-a4a1-d581b3b0d798 None admin] PATH_INFO: `/v3/projects/a1e3a96fd05742d9a58f3a78e19ef4f6` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:51.596989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e58ee-0e03-4072-a4a1-d581b3b0d798 None admin] RBAC: Authorizing `identity:get_project(project_id=a1e3a96fd05742d9a58f3a78e19ef4f6)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:51.598817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e58ee-0e03-4072-a4a1-d581b3b0d798 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:51.601296 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 213/419] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:51 2022] GET /identity/v3/projects/a1e3a96fd05742d9a58f3a78e19ef4f6 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:51.606559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca80cb-28f0-4b9c-8fd6-84f56479dd53 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:51.606559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca80cb-28f0-4b9c-8fd6-84f56479dd53 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:51.606559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca80cb-28f0-4b9c-8fd6-84f56479dd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:51.630645 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a5ca80cb-28f0-4b9c-8fd6-84f56479dd53 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:51.632472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a5ca80cb-28f0-4b9c-8fd6-84f56479dd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:51.666562 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f22b48-28c9-4f8c-99de-c2324b857bbc None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:51.710152 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88086b5a-beee-422f-bc2a-237e18fa38c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:51.710152 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88086b5a-beee-422f-bc2a-237e18fa38c7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:51.710152 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88086b5a-beee-422f-bc2a-237e18fa38c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:51.710152 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88086b5a-beee-422f-bc2a-237e18fa38c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:51.720642 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 214/420] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:51.721709 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca80cb-28f0-4b9c-8fd6-84f56479dd53 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:51.807320 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88086b5a-beee-422f-bc2a-237e18fa38c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:51.809158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88086b5a-beee-422f-bc2a-237e18fa38c7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:51.865119 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca80cb-28f0-4b9c-8fd6-84f56479dd53 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:51.885941 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:51.921100 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:51.943230 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb335e8-0000-43b6-a026-cdfded6ea318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:51.943916 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb335e8-0000-43b6-a026-cdfded6ea318 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:51.944205 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb335e8-0000-43b6-a026-cdfded6ea318 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:51.944482 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb335e8-0000-43b6-a026-cdfded6ea318 None admin] PATH_INFO: `/v3/projects/a1e3a96fd05742d9a58f3a78e19ef4f6/groups/96ee92e558184c919a319c47bfc5809d/roles/f6acb6fdca5a44de9ba00b2b0d4d3202` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:51.951692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb335e8-0000-43b6-a026-cdfded6ea318 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1e3a96fd05742d9a58f3a78e19ef4f6, group_id=96ee92e558184c919a319c47bfc5809d, role_id=f6acb6fdca5a44de9ba00b2b0d4d3202)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:51.951692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb335e8-0000-43b6-a026-cdfded6ea318 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:51.985254 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 207/421] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:51.988699 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 215/422] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Aug 23 05:59:51 2022] PUT /identity/v3/projects/a1e3a96fd05742d9a58f3a78e19ef4f6/groups/96ee92e558184c919a319c47bfc5809d/roles/f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 05:59:51.990367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88086b5a-beee-422f-bc2a-237e18fa38c7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:52.105130 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb335e8-0000-43b6-a026-cdfded6ea318 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:52.141210 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88086b5a-beee-422f-bc2a-237e18fa38c7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:52.177973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6129f-6dd5-408c-b3b7-f2b9e9f644de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:52.178680 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6129f-6dd5-408c-b3b7-f2b9e9f644de None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:52.178680 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6129f-6dd5-408c-b3b7-f2b9e9f644de None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:52.179487 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6129f-6dd5-408c-b3b7-f2b9e9f644de None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:52.180298 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6129f-6dd5-408c-b3b7-f2b9e9f644de None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:52.181745 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6129f-6dd5-408c-b3b7-f2b9e9f644de None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:52.183699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 208/423] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:51 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:52.193869 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6129f-6dd5-408c-b3b7-f2b9e9f644de None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:52.210141 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:52.266410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb335e8-0000-43b6-a026-cdfded6ea318 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:52.309688 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e4fc0-c5a9-4443-9dad-c0aba9e8b249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:52.311588 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e4fc0-c5a9-4443-9dad-c0aba9e8b249 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:52.311874 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e4fc0-c5a9-4443-9dad-c0aba9e8b249 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:52.312150 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e4fc0-c5a9-4443-9dad-c0aba9e8b249 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:52.339355 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6129f-6dd5-408c-b3b7-f2b9e9f644de None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:52.342450 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e4fc0-c5a9-4443-9dad-c0aba9e8b249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-168e4fc0-c5a9-4443-9dad-c0aba9e8b249 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:52.349076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:52.349969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:52.350777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:52.350777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:52.350777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:52.350777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:52.350777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:52.350777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:52.350777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:52.350777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:52.350777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:52.350777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:52.350777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:52.350777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:52.358598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 216/424] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 05:59:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:59:52.358598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e4fc0-c5a9-4443-9dad-c0aba9e8b249 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:52.424914 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd013050-01a4-4571-a6d7-e9deaeba3176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:52.425621 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd013050-01a4-4571-a6d7-e9deaeba3176 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:52.425903 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd013050-01a4-4571-a6d7-e9deaeba3176 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:52.426241 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd013050-01a4-4571-a6d7-e9deaeba3176 None admin] PATH_INFO: `/v3/groups/3f2aa215b75e44d2874532a46d371681` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:52.427787 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd013050-01a4-4571-a6d7-e9deaeba3176 None admin] RBAC: Authorizing `identity:get_group(group_id=3f2aa215b75e44d2874532a46d371681)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:52.429222 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd013050-01a4-4571-a6d7-e9deaeba3176 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:52.430956 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 209/425] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 23 05:59:52 2022] GET /identity/v3/groups/3f2aa215b75e44d2874532a46d371681 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:52.437588 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd013050-01a4-4571-a6d7-e9deaeba3176 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:52.461493 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:52.528567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:52.559817 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd013050-01a4-4571-a6d7-e9deaeba3176 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:52.600175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e4fc0-c5a9-4443-9dad-c0aba9e8b249 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:52.613708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83476e83-7d56-40d7-9408-3b360eb0db0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:52.614223 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476e83-7d56-40d7-9408-3b360eb0db0d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:52.614533 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476e83-7d56-40d7-9408-3b360eb0db0d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:52.614805 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476e83-7d56-40d7-9408-3b360eb0db0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:52.615539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83476e83-7d56-40d7-9408-3b360eb0db0d None admin] RBAC: Adding query filter params (group.id=3f2aa215b75e44d2874532a46d371681, role.id=a2f35dad17e94553b63d07f8b8d8f1a4, scope.project.id=bca76726c98f48f482c676027eb393d2) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:52.615843 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83476e83-7d56-40d7-9408-3b360eb0db0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:52.617289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83476e83-7d56-40d7-9408-3b360eb0db0d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:52.627087 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 210/426] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 23 05:59:52 2022] GET /identity/v3/role_assignments?group.id=3f2aa215b75e44d2874532a46d371681&scope.project.id=bca76726c98f48f482c676027eb393d2&role.id=a2f35dad17e94553b63d07f8b8d8f1a4 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:52.628409 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab2e98-9865-49ac-ba23-28fe6da86ff4 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:52.628696 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab2e98-9865-49ac-ba23-28fe6da86ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:52.628966 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab2e98-9865-49ac-ba23-28fe6da86ff4 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:52.630573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 211/427] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:52.636845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3ee82-39ac-4548-a386-a6cb841ed734 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:52.637139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3ee82-39ac-4548-a386-a6cb841ed734 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:52.637400 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3ee82-39ac-4548-a386-a6cb841ed734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:52.645473 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310cc29-160a-4ca6-81a3-4cc2b5fad411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:52.646328 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310cc29-160a-4ca6-81a3-4cc2b5fad411 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:52.646633 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310cc29-160a-4ca6-81a3-4cc2b5fad411 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:52.646901 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310cc29-160a-4ca6-81a3-4cc2b5fad411 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:52.647586 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310cc29-160a-4ca6-81a3-4cc2b5fad411 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:52.647886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310cc29-160a-4ca6-81a3-4cc2b5fad411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:52.649299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310cc29-160a-4ca6-81a3-4cc2b5fad411 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:52.659709 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 217/428] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 05:59:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:52.663626 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-3ac3ee82-39ac-4548-a386-a6cb841ed734 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:52.665130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3ac3ee82-39ac-4548-a386-a6cb841ed734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:52.672916 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310cc29-160a-4ca6-81a3-4cc2b5fad411 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:52.752381 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 212/429] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:59:52.826554 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310cc29-160a-4ca6-81a3-4cc2b5fad411 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:52.843530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:52.875875 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6864fd-7355-4388-96de-f17d09787067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:52.876602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6864fd-7355-4388-96de-f17d09787067 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:52.876773 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6864fd-7355-4388-96de-f17d09787067 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:52.876921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6864fd-7355-4388-96de-f17d09787067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:52.962542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6864fd-7355-4388-96de-f17d09787067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:52.962542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6864fd-7355-4388-96de-f17d09787067 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:53.075323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 218/430] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:53.078665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3ee82-39ac-4548-a386-a6cb841ed734 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:53.109478 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6864fd-7355-4388-96de-f17d09787067 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:53.155111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:53.179591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:53.213203 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3ee82-39ac-4548-a386-a6cb841ed734 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:53.262096 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6ce1e-fb25-45cd-8f2c-f26eee7363a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:53.262771 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6ce1e-fb25-45cd-8f2c-f26eee7363a0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:53.263048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6ce1e-fb25-45cd-8f2c-f26eee7363a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:53.263409 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6ce1e-fb25-45cd-8f2c-f26eee7363a0 None admin] PATH_INFO: `/v3/projects/bca76726c98f48f482c676027eb393d2` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:53.265150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6ce1e-fb25-45cd-8f2c-f26eee7363a0 None admin] RBAC: Authorizing `identity:get_project(project_id=bca76726c98f48f482c676027eb393d2)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:53.267713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6ce1e-fb25-45cd-8f2c-f26eee7363a0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:53.270594 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 213/431] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:53 2022] GET /identity/v3/projects/bca76726c98f48f482c676027eb393d2 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:53.276003 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6864fd-7355-4388-96de-f17d09787067 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:53.276258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6ce1e-fb25-45cd-8f2c-f26eee7363a0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:53.325353 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be71032-ebdc-4d36-8979-961d87d32f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:53.326646 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be71032-ebdc-4d36-8979-961d87d32f7a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:53.326953 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be71032-ebdc-4d36-8979-961d87d32f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:53.327225 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be71032-ebdc-4d36-8979-961d87d32f7a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:53.340671 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be71032-ebdc-4d36-8979-961d87d32f7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-0be71032-ebdc-4d36-8979-961d87d32f7a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:53.346601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:53.347502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:53.348288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:53.348288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:53.348288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:53.348288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:53.348288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:53.348288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:53.348288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:53.348288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:53.348288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:53.348288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:53.348288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 05:59:53.348288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:53.348288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 219/432] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:59:53.354919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be71032-ebdc-4d36-8979-961d87d32f7a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:53.415066 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6ce1e-fb25-45cd-8f2c-f26eee7363a0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:53.464878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7db003-afa4-4bfa-838d-95e4e6105e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:53.465716 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7db003-afa4-4bfa-838d-95e4e6105e13 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:53.466095 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7db003-afa4-4bfa-838d-95e4e6105e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:53.466336 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:53.466585 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7db003-afa4-4bfa-838d-95e4e6105e13 None admin] PATH_INFO: `/v3/groups/3f2aa215b75e44d2874532a46d371681` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:53.468458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7db003-afa4-4bfa-838d-95e4e6105e13 None admin] RBAC: Authorizing `identity:get_group(group_id=3f2aa215b75e44d2874532a46d371681)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:53.470050 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7db003-afa4-4bfa-838d-95e4e6105e13 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:53.476642 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 214/433] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 23 05:59:53 2022] GET /identity/v3/groups/3f2aa215b75e44d2874532a46d371681 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:53.477934 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7db003-afa4-4bfa-838d-95e4e6105e13 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:53.492302 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:53.502732 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be71032-ebdc-4d36-8979-961d87d32f7a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:53.547019 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11c99c-75f3-4da1-b9c7-7d1f18532037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:53.547743 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11c99c-75f3-4da1-b9c7-7d1f18532037 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:53.548288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11c99c-75f3-4da1-b9c7-7d1f18532037 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:53.548558 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11c99c-75f3-4da1-b9c7-7d1f18532037 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:53.549231 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11c99c-75f3-4da1-b9c7-7d1f18532037 None admin] RBAC: Adding query filter params (name=member) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:53.549685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11c99c-75f3-4da1-b9c7-7d1f18532037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:53.551592 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11c99c-75f3-4da1-b9c7-7d1f18532037 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:53.562683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 220/434] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 05:59:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:53.571934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11c99c-75f3-4da1-b9c7-7d1f18532037 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:53.615155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7db003-afa4-4bfa-838d-95e4e6105e13 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:53.666121 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8935c-03fa-4a20-af86-f4b6c77a596e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:53.666730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8935c-03fa-4a20-af86-f4b6c77a596e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:53.667033 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8935c-03fa-4a20-af86-f4b6c77a596e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:53.667262 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8935c-03fa-4a20-af86-f4b6c77a596e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:53.668405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8935c-03fa-4a20-af86-f4b6c77a596e None admin] RBAC: Adding query filter params (group.id=3f2aa215b75e44d2874532a46d371681, role.id=f47755365f24477f9846225c9e91de2d, scope.project.id=bca76726c98f48f482c676027eb393d2) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:53.668405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8935c-03fa-4a20-af86-f4b6c77a596e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:53.670456 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8935c-03fa-4a20-af86-f4b6c77a596e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:53.677213 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 215/435] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 23 05:59:53 2022] GET /identity/v3/role_assignments?group.id=3f2aa215b75e44d2874532a46d371681&scope.project.id=bca76726c98f48f482c676027eb393d2&role.id=f47755365f24477f9846225c9e91de2d => generated 611 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:53.678213 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddc5c5-ce6a-4276-9dc2-57321da3e462 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:53.678490 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddc5c5-ce6a-4276-9dc2-57321da3e462 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:53.681829 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddc5c5-ce6a-4276-9dc2-57321da3e462 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:53.682810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 216/436] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:53.688349 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565b401-8243-4ad0-877c-22c06c66637c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:53.688628 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565b401-8243-4ad0-877c-22c06c66637c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:53.688888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565b401-8243-4ad0-877c-22c06c66637c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:53.710656 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-c565b401-8243-4ad0-877c-22c06c66637c None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:53.711878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c565b401-8243-4ad0-877c-22c06c66637c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:53.726889 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11c99c-75f3-4da1-b9c7-7d1f18532037 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:53.769474 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af04a8-6cd2-4bea-81ba-c9804c27de57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:53.770088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af04a8-6cd2-4bea-81ba-c9804c27de57 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:53.770362 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af04a8-6cd2-4bea-81ba-c9804c27de57 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:53.770682 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af04a8-6cd2-4bea-81ba-c9804c27de57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:53.773673 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:53.811971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 217/437] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:53.856132 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af04a8-6cd2-4bea-81ba-c9804c27de57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:53.858169 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af04a8-6cd2-4bea-81ba-c9804c27de57 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:53.970872 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 221/438] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 05:59:53.978432 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565b401-8243-4ad0-877c-22c06c66637c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:53.981449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:54.085144 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af04a8-6cd2-4bea-81ba-c9804c27de57 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:54.103541 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565b401-8243-4ad0-877c-22c06c66637c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:54.132666 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:54.153858 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c23dd-544d-452a-b0c5-23870a59d35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:54.155405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c23dd-544d-452a-b0c5-23870a59d35c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:54.156123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c23dd-544d-452a-b0c5-23870a59d35c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:54.156538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c23dd-544d-452a-b0c5-23870a59d35c None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:54.157840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c23dd-544d-452a-b0c5-23870a59d35c None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:54.160225 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c23dd-544d-452a-b0c5-23870a59d35c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:54.162607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 218/439] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:53 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:54.168198 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c23dd-544d-452a-b0c5-23870a59d35c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:54.220480 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af04a8-6cd2-4bea-81ba-c9804c27de57 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:54.254245 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:54.259012 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05c76e-ba18-4363-a032-57bca7869616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:54.259643 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05c76e-ba18-4363-a032-57bca7869616 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:54.259916 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05c76e-ba18-4363-a032-57bca7869616 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:54.260222 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05c76e-ba18-4363-a032-57bca7869616 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:54.270900 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05c76e-ba18-4363-a032-57bca7869616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-3d05c76e-ba18-4363-a032-57bca7869616 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:54.275576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:54.276546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:54.277346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:54.277346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:54.277346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:54.277346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:54.277346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:54.277346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:54.277346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:54.277346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:54.277346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:54.277346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:54.277346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 05:59:54.277346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 05:59:54.278655 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 222/440] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 05:59:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:54.280723 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d466b24-b098-4d1f-ab8f-8f44718f5d73 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:54.280996 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d466b24-b098-4d1f-ab8f-8f44718f5d73 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:54.281267 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d466b24-b098-4d1f-ab8f-8f44718f5d73 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:54.282860 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 223/441] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:54.283542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d466b24-b098-4d1f-ab8f-8f44718f5d73 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:54.287770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c23dd-544d-452a-b0c5-23870a59d35c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:54.322406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6191d42-4c94-4998-977d-7487deca598c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:54.323050 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6191d42-4c94-4998-977d-7487deca598c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:54.323323 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6191d42-4c94-4998-977d-7487deca598c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:54.323586 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6191d42-4c94-4998-977d-7487deca598c None admin] PATH_INFO: `/v3/groups/3f2aa215b75e44d2874532a46d371681` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:54.324701 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6191d42-4c94-4998-977d-7487deca598c None admin] RBAC: Authorizing `identity:get_group(group_id=3f2aa215b75e44d2874532a46d371681)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:54.326176 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6191d42-4c94-4998-977d-7487deca598c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:54.327775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 219/442] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 23 05:59:54 2022] GET /identity/v3/groups/3f2aa215b75e44d2874532a46d371681 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:54.331710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428260fe-6336-4a3a-9bbc-5d5fefc616e4 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:54.331992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428260fe-6336-4a3a-9bbc-5d5fefc616e4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:54.332263 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428260fe-6336-4a3a-9bbc-5d5fefc616e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:54.350460 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-428260fe-6336-4a3a-9bbc-5d5fefc616e4 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:54.351669 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-428260fe-6336-4a3a-9bbc-5d5fefc616e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:54.379704 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:54.387836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d466b24-b098-4d1f-ab8f-8f44718f5d73 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:54.418159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 220/443] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:54.420843 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428260fe-6336-4a3a-9bbc-5d5fefc616e4 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:54.423232 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f97dc1-491a-428f-845f-df085635d01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:54.423839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f97dc1-491a-428f-845f-df085635d01f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:54.424117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f97dc1-491a-428f-845f-df085635d01f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:54.424391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f97dc1-491a-428f-845f-df085635d01f None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:54.425064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f97dc1-491a-428f-845f-df085635d01f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:54.425369 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f97dc1-491a-428f-845f-df085635d01f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:54.426774 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f97dc1-491a-428f-845f-df085635d01f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:54.435732 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 224/444] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 05:59:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:54.441343 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f97dc1-491a-428f-845f-df085635d01f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:54.500434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:54.535769 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428260fe-6336-4a3a-9bbc-5d5fefc616e4 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:54.548505 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f97dc1-491a-428f-845f-df085635d01f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:54.571032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57067a91-5367-4997-affb-c929af5efaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:54.571629 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57067a91-5367-4997-affb-c929af5efaf6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:54.571900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57067a91-5367-4997-affb-c929af5efaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:54.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57067a91-5367-4997-affb-c929af5efaf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:54.572877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57067a91-5367-4997-affb-c929af5efaf6 None admin] RBAC: Adding query filter params (group.id=3f2aa215b75e44d2874532a46d371681, role.id=a2f35dad17e94553b63d07f8b8d8f1a4, scope.project.id=b4d0c8983a01433eb5c05fa508bede98) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:54.573177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57067a91-5367-4997-affb-c929af5efaf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:54.574620 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57067a91-5367-4997-affb-c929af5efaf6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:54.582019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 221/445] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 23 05:59:54 2022] GET /identity/v3/role_assignments?group.id=3f2aa215b75e44d2874532a46d371681&scope.project.id=b4d0c8983a01433eb5c05fa508bede98&role.id=a2f35dad17e94553b63d07f8b8d8f1a4 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:54.587180 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfa4e9-4bb3-4e3b-862a-9b6de946381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:54.587915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfa4e9-4bb3-4e3b-862a-9b6de946381d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:54.588261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfa4e9-4bb3-4e3b-862a-9b6de946381d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:54.588607 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfa4e9-4bb3-4e3b-862a-9b6de946381d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:54.621971 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:54.674749 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfa4e9-4bb3-4e3b-862a-9b6de946381d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:54.676218 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfa4e9-4bb3-4e3b-862a-9b6de946381d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:54.688496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57067a91-5367-4997-affb-c929af5efaf6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:54.797517 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 225/446] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 05:59:54.803150 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfa4e9-4bb3-4e3b-862a-9b6de946381d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:54.813499 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57067a91-5367-4997-affb-c929af5efaf6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:54.849687 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:54.858823 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c01a1-9022-46bb-9e0f-8ef0bdc3467d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:54.859672 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c01a1-9022-46bb-9e0f-8ef0bdc3467d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:54.860018 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c01a1-9022-46bb-9e0f-8ef0bdc3467d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:54.860295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c01a1-9022-46bb-9e0f-8ef0bdc3467d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:54.875819 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:54.876043 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c01a1-9022-46bb-9e0f-8ef0bdc3467d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-923c01a1-9022-46bb-9e0f-8ef0bdc3467d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 05:59:54.881526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:54.882828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:54.884352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:54.884352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:54.884352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:54.884352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:54.884352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 05:59:54.884352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 05:59:54.884352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 05:59:54.884352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 05:59:54.884352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 05:59:54.884352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 05:59:54.884352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 05:59:54.884352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:54.884352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 222/447] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 05:59:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 05:59:54.889373 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c01a1-9022-46bb-9e0f-8ef0bdc3467d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:54.934717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfa4e9-4bb3-4e3b-862a-9b6de946381d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:54.976593 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a473cea-930a-47f0-aebb-fcd992660010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:54.977229 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a473cea-930a-47f0-aebb-fcd992660010 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:54.977480 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a473cea-930a-47f0-aebb-fcd992660010 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:54.977748 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a473cea-930a-47f0-aebb-fcd992660010 None admin] PATH_INFO: `/v3/projects/b4d0c8983a01433eb5c05fa508bede98` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:54.978884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a473cea-930a-47f0-aebb-fcd992660010 None admin] RBAC: Authorizing `identity:get_project(project_id=b4d0c8983a01433eb5c05fa508bede98)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:54.980311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a473cea-930a-47f0-aebb-fcd992660010 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:54.981988 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 226/448] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:54 2022] GET /identity/v3/projects/b4d0c8983a01433eb5c05fa508bede98 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:54.988685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a473cea-930a-47f0-aebb-fcd992660010 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:55.015422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c01a1-9022-46bb-9e0f-8ef0bdc3467d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:55.056561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69179145-f566-41f8-9d82-f7c68cae7cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:55.057105 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69179145-f566-41f8-9d82-f7c68cae7cef None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:55.057388 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69179145-f566-41f8-9d82-f7c68cae7cef None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:55.057631 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69179145-f566-41f8-9d82-f7c68cae7cef None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:55.058424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69179145-f566-41f8-9d82-f7c68cae7cef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:55.059000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69179145-f566-41f8-9d82-f7c68cae7cef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:55.061845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69179145-f566-41f8-9d82-f7c68cae7cef None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:55.071651 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 223/449] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 05:59:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:55.077931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69179145-f566-41f8-9d82-f7c68cae7cef None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:55.096730 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a473cea-930a-47f0-aebb-fcd992660010 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:55.128219 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:55.141651 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ac8a0-23fa-4a12-bd9c-1cf8f114f514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:55.142288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ac8a0-23fa-4a12-bd9c-1cf8f114f514 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:55.142603 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ac8a0-23fa-4a12-bd9c-1cf8f114f514 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:55.142826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:55.142967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ac8a0-23fa-4a12-bd9c-1cf8f114f514 None admin] PATH_INFO: `/v3/groups/3f2aa215b75e44d2874532a46d371681` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:55.144043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ac8a0-23fa-4a12-bd9c-1cf8f114f514 None admin] RBAC: Authorizing `identity:get_group(group_id=3f2aa215b75e44d2874532a46d371681)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:55.145870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ac8a0-23fa-4a12-bd9c-1cf8f114f514 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:55.152581 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 227/450] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 23 05:59:54 2022] GET /identity/v3/groups/3f2aa215b75e44d2874532a46d371681 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:55.153466 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ac8a0-23fa-4a12-bd9c-1cf8f114f514 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:55.193700 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69179145-f566-41f8-9d82-f7c68cae7cef None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:55.233060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aca333-0766-4215-937f-cf136c8dcccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:55.233749 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aca333-0766-4215-937f-cf136c8dcccd None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:55.234020 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aca333-0766-4215-937f-cf136c8dcccd None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:55.234285 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aca333-0766-4215-937f-cf136c8dcccd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:55.281841 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ac8a0-23fa-4a12-bd9c-1cf8f114f514 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:55.312870 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aca333-0766-4215-937f-cf136c8dcccd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:55.314341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aca333-0766-4215-937f-cf136c8dcccd None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:55.320193 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabcbb2-4a08-4523-9682-305d2fbcb639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:55.320792 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabcbb2-4a08-4523-9682-305d2fbcb639 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:55.321066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabcbb2-4a08-4523-9682-305d2fbcb639 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:55.321334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabcbb2-4a08-4523-9682-305d2fbcb639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:55.322065 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabcbb2-4a08-4523-9682-305d2fbcb639 None admin] RBAC: Adding query filter params (group.id=3f2aa215b75e44d2874532a46d371681, role.id=f47755365f24477f9846225c9e91de2d, scope.project.id=b4d0c8983a01433eb5c05fa508bede98) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:55.322377 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabcbb2-4a08-4523-9682-305d2fbcb639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:55.323850 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabcbb2-4a08-4523-9682-305d2fbcb639 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:55.331016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 228/451] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 23 05:59:55 2022] GET /identity/v3/role_assignments?group.id=3f2aa215b75e44d2874532a46d371681&scope.project.id=b4d0c8983a01433eb5c05fa508bede98&role.id=f47755365f24477f9846225c9e91de2d => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 05:59:55.384951 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:55.442542 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 224/452] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 05:59:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 05:59:55.448624 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabcbb2-4a08-4523-9682-305d2fbcb639 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:55.514691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:55.573752 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabcbb2-4a08-4523-9682-305d2fbcb639 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:55.615251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb55d4-ada9-424c-b8d7-434e3cb29f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:55.615879 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb55d4-ada9-424c-b8d7-434e3cb29f21 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:55.616154 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb55d4-ada9-424c-b8d7-434e3cb29f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:55.616422 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb55d4-ada9-424c-b8d7-434e3cb29f21 None admin] PATH_INFO: `/v3/projects/a1e3a96fd05742d9a58f3a78e19ef4f6` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:55.617797 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb55d4-ada9-424c-b8d7-434e3cb29f21 None admin] RBAC: Authorizing `identity:get_project(project_id=a1e3a96fd05742d9a58f3a78e19ef4f6)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:55.620206 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb55d4-ada9-424c-b8d7-434e3cb29f21 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:55.622545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 229/453] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 05:59:55 2022] GET /identity/v3/projects/a1e3a96fd05742d9a58f3a78e19ef4f6 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:55.629259 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aca333-0766-4215-937f-cf136c8dcccd None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:55.759691 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aca333-0766-4215-937f-cf136c8dcccd None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:55.800391 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcf71d-5d4c-4c3e-a772-5aa81ad8f50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:55.801036 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcf71d-5d4c-4c3e-a772-5aa81ad8f50a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:55.801314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcf71d-5d4c-4c3e-a772-5aa81ad8f50a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:55.801583 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcf71d-5d4c-4c3e-a772-5aa81ad8f50a None admin] PATH_INFO: `/v3/groups/96ee92e558184c919a319c47bfc5809d` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:55.802724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcf71d-5d4c-4c3e-a772-5aa81ad8f50a None admin] RBAC: Authorizing `identity:get_group(group_id=96ee92e558184c919a319c47bfc5809d)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:55.804307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcf71d-5d4c-4c3e-a772-5aa81ad8f50a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:55.805899 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 225/454] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Aug 23 05:59:55 2022] GET /identity/v3/groups/96ee92e558184c919a319c47bfc5809d => generated 229 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:55.812626 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb55d4-ada9-424c-b8d7-434e3cb29f21 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:55.931057 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb55d4-ada9-424c-b8d7-434e3cb29f21 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:55.953575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:55.963957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d7e8a-3e98-4e39-b250-946f6895c13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:55.964551 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d7e8a-3e98-4e39-b250-946f6895c13c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:55.964826 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d7e8a-3e98-4e39-b250-946f6895c13c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:55.965093 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d7e8a-3e98-4e39-b250-946f6895c13c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:55.965813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d7e8a-3e98-4e39-b250-946f6895c13c None admin] RBAC: Adding query filter params (group.id=96ee92e558184c919a319c47bfc5809d, role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=a1e3a96fd05742d9a58f3a78e19ef4f6) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:55.966116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d7e8a-3e98-4e39-b250-946f6895c13c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:55.969026 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d7e8a-3e98-4e39-b250-946f6895c13c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:55.974953 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 230/455] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Aug 23 05:59:55 2022] GET /identity/v3/role_assignments?group.id=96ee92e558184c919a319c47bfc5809d&scope.project.id=a1e3a96fd05742d9a58f3a78e19ef4f6&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:58.281828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469669e-fb8b-49fe-ba2c-9822dda25de2 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:58.282706 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469669e-fb8b-49fe-ba2c-9822dda25de2 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:58.283116 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469669e-fb8b-49fe-ba2c-9822dda25de2 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:58.284839 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 226/456] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:58.290729 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5382d-b5f8-4a25-8c88-6d3fc86348c5 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:58.291656 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5382d-b5f8-4a25-8c88-6d3fc86348c5 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:58.292141 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5382d-b5f8-4a25-8c88-6d3fc86348c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:58.326270 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1ec5382d-b5f8-4a25-8c88-6d3fc86348c5 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:58.328840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1ec5382d-b5f8-4a25-8c88-6d3fc86348c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:58.436642 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 231/457] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:58.611625 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d703b5-9678-4d8c-b2c6-b7c4e4534e2f None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:58.613466 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d703b5-9678-4d8c-b2c6-b7c4e4534e2f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:58.613466 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d703b5-9678-4d8c-b2c6-b7c4e4534e2f None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:58.613903 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 227/458] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:58.619263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bad49-d40f-4bec-8e80-c0921da388d0 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:58.619554 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bad49-d40f-4bec-8e80-c0921da388d0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:58.619866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bad49-d40f-4bec-8e80-c0921da388d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:58.646854 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-892bad49-d40f-4bec-8e80-c0921da388d0 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:58.648053 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-892bad49-d40f-4bec-8e80-c0921da388d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:58.730019 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 232/459] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:58.880203 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d703b5-9678-4d8c-b2c6-b7c4e4534e2f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:58.912125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:59.017762 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057e9a0-e1f5-45fb-b353-a39be3ab223c None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:59.018598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057e9a0-e1f5-45fb-b353-a39be3ab223c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:59.020261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057e9a0-e1f5-45fb-b353-a39be3ab223c None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:59.023646 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 233/460] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:59.032108 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626caa9b-6542-4f2b-8458-66445cc7e202 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:59.032668 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626caa9b-6542-4f2b-8458-66445cc7e202 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:59.033155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626caa9b-6542-4f2b-8458-66445cc7e202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:59.034391 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d703b5-9678-4d8c-b2c6-b7c4e4534e2f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:59.072256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-626caa9b-6542-4f2b-8458-66445cc7e202 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:59.073713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-626caa9b-6542-4f2b-8458-66445cc7e202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:59.100914 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72899e78-9089-4fe5-abb1-0201902c16a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:59.101559 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72899e78-9089-4fe5-abb1-0201902c16a0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:59.101893 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72899e78-9089-4fe5-abb1-0201902c16a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:59.102167 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72899e78-9089-4fe5-abb1-0201902c16a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:59.117925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72899e78-9089-4fe5-abb1-0201902c16a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:59.119771 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72899e78-9089-4fe5-abb1-0201902c16a0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:59.134024 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-72899e78-9089-4fe5-abb1-0201902c16a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 05:59:59.140554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:59.141454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 05:59:59.142300 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 05:59:59.142300 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 05:59:59.142300 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:59.148457 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 228/461] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:59.148457 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72899e78-9089-4fe5-abb1-0201902c16a0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:59.173000 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 234/462] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 05:59:59.174207 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626caa9b-6542-4f2b-8458-66445cc7e202 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:59.262863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:59.291478 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72899e78-9089-4fe5-abb1-0201902c16a0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:59.308621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626caa9b-6542-4f2b-8458-66445cc7e202 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:59.331984 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8074c-fb96-4f69-b74c-f9db35872130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:59.332866 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8074c-fb96-4f69-b74c-f9db35872130 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:59.333237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8074c-fb96-4f69-b74c-f9db35872130 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:59.333621 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8074c-fb96-4f69-b74c-f9db35872130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:59.345239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa4c44-0ded-4946-b70a-ea49ae877c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:59.346093 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa4c44-0ded-4946-b70a-ea49ae877c08 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:59.346197 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8074c-fb96-4f69-b74c-f9db35872130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:59.346701 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa4c44-0ded-4946-b70a-ea49ae877c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:59.347149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa4c44-0ded-4946-b70a-ea49ae877c08 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:59.347784 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8074c-fb96-4f69-b74c-f9db35872130 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:59.348145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa4c44-0ded-4946-b70a-ea49ae877c08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:59.348612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa4c44-0ded-4946-b70a-ea49ae877c08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:59.350140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa4c44-0ded-4946-b70a-ea49ae877c08 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-81a8074c-fb96-4f69-b74c-f9db35872130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 05:59:59.363729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:59.364662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 05:59:59.365571 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 05:59:59.365571 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 05:59:59.365571 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:59.370009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 229/463] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:59.371344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30396bfd-e10f-47f0-9a01-a217ddf6474e None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:59.371626 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30396bfd-e10f-47f0-9a01-a217ddf6474e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:59.371898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30396bfd-e10f-47f0-9a01-a217ddf6474e None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:59.372901 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 235/464] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 05:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:59.373439 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 230/465] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:59.378349 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa4c44-0ded-4946-b70a-ea49ae877c08 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:59.378942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30396bfd-e10f-47f0-9a01-a217ddf6474e None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:59.424404 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:59.497975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30396bfd-e10f-47f0-9a01-a217ddf6474e None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:59.501365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa4c44-0ded-4946-b70a-ea49ae877c08 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:59.523895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:59.534449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15ddf3-8a1b-4fb9-8e66-8367627a8865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:59.535091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15ddf3-8a1b-4fb9-8e66-8367627a8865 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:59.535368 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15ddf3-8a1b-4fb9-8e66-8367627a8865 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:59.535624 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15ddf3-8a1b-4fb9-8e66-8367627a8865 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:59.536440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15ddf3-8a1b-4fb9-8e66-8367627a8865 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:59.537917 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8012bc-edf9-4cee-ad9a-9daef1806243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:59.538286 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15ddf3-8a1b-4fb9-8e66-8367627a8865 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:59.538541 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8012bc-edf9-4cee-ad9a-9daef1806243 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:59.538815 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8012bc-edf9-4cee-ad9a-9daef1806243 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:59.539083 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8012bc-edf9-4cee-ad9a-9daef1806243 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:59.539722 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8012bc-edf9-4cee-ad9a-9daef1806243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 05:59:59.540026 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8012bc-edf9-4cee-ad9a-9daef1806243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:59.541367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8012bc-edf9-4cee-ad9a-9daef1806243 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:59.554661 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 236/466] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 05:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 05:59:59.555884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cd432-be6e-4cf4-8711-8b74edf797b9 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:59.556281 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cd432-be6e-4cf4-8711-8b74edf797b9 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:59.556564 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cd432-be6e-4cf4-8711-8b74edf797b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:59.570737 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 231/467] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 05:59:59 2022] POST /identity/v3/users => generated 246 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 05:59:59.571723 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15ddf3-8a1b-4fb9-8e66-8367627a8865 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:59.580649 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-960cd432-be6e-4cf4-8711-8b74edf797b9 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:59.582021 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-960cd432-be6e-4cf4-8711-8b74edf797b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 05:59:59.671021 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 237/468] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 05:59:59.672204 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960cd432-be6e-4cf4-8711-8b74edf797b9 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:59.713955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15ddf3-8a1b-4fb9-8e66-8367627a8865 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:59.719859 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:59.759694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed95efb-99c4-44d0-ba4c-3f5ac7cec9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:59.760696 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed95efb-99c4-44d0-ba4c-3f5ac7cec9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:59.761186 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed95efb-99c4-44d0-ba4c-3f5ac7cec9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:59.761744 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed95efb-99c4-44d0-ba4c-3f5ac7cec9c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:59.775077 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed95efb-99c4-44d0-ba4c-3f5ac7cec9c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:59.776942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed95efb-99c4-44d0-ba4c-3f5ac7cec9c3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-bed95efb-99c4-44d0-ba4c-3f5ac7cec9c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 05:59:59.792567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 05:59:59.793486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 05:59:59.794867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 05:59:59.794867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 05:59:59.794867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 05:59:59.797159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 232/469] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 05:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 05:59:59.803415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960cd432-be6e-4cf4-8711-8b74edf797b9 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:59.804610 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed95efb-99c4-44d0-ba4c-3f5ac7cec9c3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 05:59:59.850443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece42e36-dab0-45f7-9d80-016446afda73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 05:59:59.851079 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece42e36-dab0-45f7-9d80-016446afda73 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:59.851354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 05:59:59.851467 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece42e36-dab0-45f7-9d80-016446afda73 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:59.851618 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece42e36-dab0-45f7-9d80-016446afda73 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:59.852301 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece42e36-dab0-45f7-9d80-016446afda73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 05:59:59.854114 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece42e36-dab0-45f7-9d80-016446afda73 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 05:59:59.882547 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 238/470] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 05:59:59 2022] POST /identity/v3/users => generated 248 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 05:59:59.910791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d29425-cb59-4aaf-93af-4ecebf39b3a2 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:59.911087 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d29425-cb59-4aaf-93af-4ecebf39b3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:59.911381 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d29425-cb59-4aaf-93af-4ecebf39b3a2 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:59.914247 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 239/471] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 05:59:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 05:59:59.922372 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678324f-8884-41b4-8566-5f151288d587 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 05:59:59.922728 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678324f-8884-41b4-8566-5f151288d587 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 05:59:59.922958 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678324f-8884-41b4-8566-5f151288d587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 05:59:59.950269 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b678324f-8884-41b4-8566-5f151288d587 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 05:59:59.951010 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed95efb-99c4-44d0-ba4c-3f5ac7cec9c3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 05:59:59.951110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b678324f-8884-41b4-8566-5f151288d587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:00.003467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab061dc5-e5a8-4184-ae35-3dcbfe81051c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:00.004189 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab061dc5-e5a8-4184-ae35-3dcbfe81051c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:00.004634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab061dc5-e5a8-4184-ae35-3dcbfe81051c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:00.004915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab061dc5-e5a8-4184-ae35-3dcbfe81051c None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:00.005578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab061dc5-e5a8-4184-ae35-3dcbfe81051c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:00.005881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab061dc5-e5a8-4184-ae35-3dcbfe81051c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:00.008059 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab061dc5-e5a8-4184-ae35-3dcbfe81051c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:00.030602 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 233/472] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 05:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:00.033214 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab061dc5-e5a8-4184-ae35-3dcbfe81051c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:00.045544 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 240/473] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 05:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:00.046789 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678324f-8884-41b4-8566-5f151288d587 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:00.072165 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:00.182208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678324f-8884-41b4-8566-5f151288d587 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:00.195457 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:00.205750 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab061dc5-e5a8-4184-ae35-3dcbfe81051c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:00.228813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f7f43-449c-41c2-ad18-e5b69d45bcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:00.229839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f7f43-449c-41c2-ad18-e5b69d45bcb0 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:00.230651 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f7f43-449c-41c2-ad18-e5b69d45bcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:00.231101 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f7f43-449c-41c2-ad18-e5b69d45bcb0 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:00.233026 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f7f43-449c-41c2-ad18-e5b69d45bcb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:00.235380 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f7f43-449c-41c2-ad18-e5b69d45bcb0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:00.245650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173203c-ad19-4008-9a87-447484ab657c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:00.246364 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173203c-ad19-4008-9a87-447484ab657c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:00.246761 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173203c-ad19-4008-9a87-447484ab657c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:00.247094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173203c-ad19-4008-9a87-447484ab657c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:00.263972 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173203c-ad19-4008-9a87-447484ab657c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:00.265768 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173203c-ad19-4008-9a87-447484ab657c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:00.267271 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 241/474] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 06:00:00 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-6173203c-ad19-4008-9a87-447484ab657c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:00.283199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:00.284171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:00.284990 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:00.284990 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:00.284990 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:00.287230 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 234/475] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:00.294946 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f7f43-449c-41c2-ad18-e5b69d45bcb0 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:00.384810 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:00.391775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173203c-ad19-4008-9a87-447484ab657c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:00.434082 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f7f43-449c-41c2-ad18-e5b69d45bcb0 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:00.488390 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87951ce2-1f57-48db-9838-98eda3ddc87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:00.489274 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87951ce2-1f57-48db-9838-98eda3ddc87f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:00.489553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87951ce2-1f57-48db-9838-98eda3ddc87f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:00.489826 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87951ce2-1f57-48db-9838-98eda3ddc87f None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:00.491212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87951ce2-1f57-48db-9838-98eda3ddc87f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:00.491518 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87951ce2-1f57-48db-9838-98eda3ddc87f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:00.493460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87951ce2-1f57-48db-9838-98eda3ddc87f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:00.511369 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 242/476] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:00.517602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87951ce2-1f57-48db-9838-98eda3ddc87f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:00.546032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173203c-ad19-4008-9a87-447484ab657c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:00.582380 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c90eb8-185a-4999-9e7b-2edffe44321f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:00.583014 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c90eb8-185a-4999-9e7b-2edffe44321f None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:00.583290 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c90eb8-185a-4999-9e7b-2edffe44321f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:00.583555 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c90eb8-185a-4999-9e7b-2edffe44321f None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:00.584290 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c90eb8-185a-4999-9e7b-2edffe44321f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:00.585648 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c90eb8-185a-4999-9e7b-2edffe44321f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:00.594267 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.sql.core [None req-56c90eb8-185a-4999-9e7b-2edffe44321f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 06:00:00.594267 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 23 06:00:00.594267 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [parameters: {'id': 'ca1dfaa1095c41e5a75af0871c8b37f9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 23 06:00:00.594267 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81747) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 06:00:00.613920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-56c90eb8-185a-4999-9e7b-2edffe44321f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 23 06:00:00.613920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:00.613920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 06:00:00.613920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 06:00:00.613920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 06:00:00.613920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 06:00:00.613920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 06:00:00.613920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 06:00:00.613920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 06:00:00.613920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application conn.query(q) Aug 23 06:00:00.613920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 06:00:00.613920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 06:00:00.613920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 06:00:00.613920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application result.read() Aug 23 06:00:00.613920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return ref.to_dict() Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 06:00:00.614878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application next(self.gen) Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application yield resource Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application next(self.gen) Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise self.value Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application session.commit() Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 06:00:00.615763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self.session.flush() Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application compat.raise_( Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 06:00:00.616614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 06:00:00.617719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application conn.query(q) Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application result.read() Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 06:00:00.624056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [parameters: {'id': 'ca1dfaa1095c41e5a75af0871c8b37f9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:00.626498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 23 06:00:00.628463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:00.630407 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 235/477] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Aug 23 06:00:00 2022] POST /identity/v3/roles => generated 144 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:00.630407 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c90eb8-185a-4999-9e7b-2edffe44321f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:00.647442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:00.652466 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87951ce2-1f57-48db-9838-98eda3ddc87f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:00.696722 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:00.707216 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bba04-cca6-471f-8a69-40cf3009987c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:00.707830 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bba04-cca6-471f-8a69-40cf3009987c None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:00.708111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bba04-cca6-471f-8a69-40cf3009987c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:00.708384 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bba04-cca6-471f-8a69-40cf3009987c None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:00.709299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270bba04-cca6-471f-8a69-40cf3009987c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:00.711689 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270bba04-cca6-471f-8a69-40cf3009987c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:00.741271 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 243/478] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 06:00:00 2022] POST /identity/v3/users => generated 249 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 06:00:00.781843 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c90eb8-185a-4999-9e7b-2edffe44321f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:00.826731 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b154c-a6fd-47a7-96b9-4bddef0db527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:00.827369 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b154c-a6fd-47a7-96b9-4bddef0db527 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:00.827649 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b154c-a6fd-47a7-96b9-4bddef0db527 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:00.827917 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b154c-a6fd-47a7-96b9-4bddef0db527 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:00.840067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b154c-a6fd-47a7-96b9-4bddef0db527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-487b154c-a6fd-47a7-96b9-4bddef0db527 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:00.847023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:00.847955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:00.848734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:00.848734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:00.848734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:00.848734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:00.848734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:00.848734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:00.848734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:00.848734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:00.848734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:00.848734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:00.848734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 06:00:00.848734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:00.851243 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 236/479] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 06:00:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:00.854542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bba04-cca6-471f-8a69-40cf3009987c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:00.984172 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bba04-cca6-471f-8a69-40cf3009987c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:01.023973 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e8374-080f-4dea-8f54-b9debe6deb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:01.024605 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e8374-080f-4dea-8f54-b9debe6deb36 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:01.024883 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e8374-080f-4dea-8f54-b9debe6deb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:01.025156 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e8374-080f-4dea-8f54-b9debe6deb36 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:01.025842 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e8374-080f-4dea-8f54-b9debe6deb36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:01.026209 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e8374-080f-4dea-8f54-b9debe6deb36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:01.027796 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e8374-080f-4dea-8f54-b9debe6deb36 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:01.042790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 244/480] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:02.051130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd603519-90b7-4a47-be23-122182c88df3 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:02.051771 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd603519-90b7-4a47-be23-122182c88df3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:02.051771 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd603519-90b7-4a47-be23-122182c88df3 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:02.053239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 237/481] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:02.071114 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06e86c-3dae-4c26-8023-0cb8f71cd4ba None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:02.071683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06e86c-3dae-4c26-8023-0cb8f71cd4ba None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:02.071683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06e86c-3dae-4c26-8023-0cb8f71cd4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:02.095566 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-0f06e86c-3dae-4c26-8023-0cb8f71cd4ba None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:02.096762 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0f06e86c-3dae-4c26-8023-0cb8f71cd4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:02.150611 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:02.181102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 245/482] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:02.517605 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6afc83-17cf-430b-882b-3423386ab223 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:02.518121 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6afc83-17cf-430b-882b-3423386ab223 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:02.518547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6afc83-17cf-430b-882b-3423386ab223 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:02.520283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 238/483] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:02.525395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c534f0-12dd-4770-8fd0-8a2ae914c002 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:02.525847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c534f0-12dd-4770-8fd0-8a2ae914c002 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:02.526289 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c534f0-12dd-4770-8fd0-8a2ae914c002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:02.548571 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c8c534f0-12dd-4770-8fd0-8a2ae914c002 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:02.549973 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c8c534f0-12dd-4770-8fd0-8a2ae914c002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:02.622255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 246/484] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:02.637296 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6afc83-17cf-430b-882b-3423386ab223 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:02.713686 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:02.756886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6afc83-17cf-430b-882b-3423386ab223 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:02.782048 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d848-1c6c-4447-91dd-221ffc0dd601 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:02.782715 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d848-1c6c-4447-91dd-221ffc0dd601 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:02.783185 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d848-1c6c-4447-91dd-221ffc0dd601 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:02.785228 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 247/485] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:00:02.790763 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e25a1-9679-4d64-8886-489ee1a21503 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:02.791319 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e25a1-9679-4d64-8886-489ee1a21503 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:02.791786 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e25a1-9679-4d64-8886-489ee1a21503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:02.793595 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753eb83a-7a2a-4585-9da2-822c5b9195ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:02.795017 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753eb83a-7a2a-4585-9da2-822c5b9195ce None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:02.795233 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753eb83a-7a2a-4585-9da2-822c5b9195ce None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:02.795511 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753eb83a-7a2a-4585-9da2-822c5b9195ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:02.814543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753eb83a-7a2a-4585-9da2-822c5b9195ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-753eb83a-7a2a-4585-9da2-822c5b9195ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:02.821959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:02.823671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:02.825293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:02.825293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:02.825293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:02.825293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:02.825293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:02.825293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:02.825293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:02.825293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:02.825293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:02.825293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:02.825293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:02.825293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:02.827216 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c45e25a1-9679-4d64-8886-489ee1a21503 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:02.828075 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c45e25a1-9679-4d64-8886-489ee1a21503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:02.834661 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 239/486] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:02.839252 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753eb83a-7a2a-4585-9da2-822c5b9195ce None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:02.973463 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 248/487] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:00:03.010090 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753eb83a-7a2a-4585-9da2-822c5b9195ce None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:03.026989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e25a1-9679-4d64-8886-489ee1a21503 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:03.068524 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:03.074872 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0d8a2-ceb4-46c3-b721-afe81ddc23d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:03.076104 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0d8a2-ceb4-46c3-b721-afe81ddc23d0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:03.076380 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0d8a2-ceb4-46c3-b721-afe81ddc23d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:03.076654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0d8a2-ceb4-46c3-b721-afe81ddc23d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:03.077363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0d8a2-ceb4-46c3-b721-afe81ddc23d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:03.077877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0d8a2-ceb4-46c3-b721-afe81ddc23d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:03.079506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0d8a2-ceb4-46c3-b721-afe81ddc23d0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:03.100363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 240/488] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:03.102872 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:03.106904 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0d8a2-ceb4-46c3-b721-afe81ddc23d0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:03.231412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e25a1-9679-4d64-8886-489ee1a21503 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:03.279860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0d8a2-ceb4-46c3-b721-afe81ddc23d0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:03.285430 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f396fab-d545-4e17-adbc-29bcbf3643f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:03.286060 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f396fab-d545-4e17-adbc-29bcbf3643f9 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:03.286332 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f396fab-d545-4e17-adbc-29bcbf3643f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:03.286627 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f396fab-d545-4e17-adbc-29bcbf3643f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:03.298413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f396fab-d545-4e17-adbc-29bcbf3643f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-4f396fab-d545-4e17-adbc-29bcbf3643f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:03.303350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:03.304280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:03.305737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:03.305737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:03.305737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:03.305737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:03.305737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:03.305737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:03.305737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:03.305737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:03.305737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:03.305737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:03.305737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:03.305737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:03.305737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 249/489] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:03.305737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f396fab-d545-4e17-adbc-29bcbf3643f9 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:03.323543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c80bbb-caa0-4e31-8d63-23c21a4a327e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:03.324524 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c80bbb-caa0-4e31-8d63-23c21a4a327e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:03.325077 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c80bbb-caa0-4e31-8d63-23c21a4a327e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:03.325592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c80bbb-caa0-4e31-8d63-23c21a4a327e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:03.339469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c80bbb-caa0-4e31-8d63-23c21a4a327e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:03.342012 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c80bbb-caa0-4e31-8d63-23c21a4a327e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-67c80bbb-caa0-4e31-8d63-23c21a4a327e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:03.361268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:03.362237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:03.363094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:03.363094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:03.363094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:03.370003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 241/490] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:03.371159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c80bbb-caa0-4e31-8d63-23c21a4a327e None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:03.414949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:03.419084 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f396fab-d545-4e17-adbc-29bcbf3643f9 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:03.425180 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:03.463212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f90bae-f229-4b4b-a63f-33df297561f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:03.463843 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f90bae-f229-4b4b-a63f-33df297561f2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:03.464106 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f90bae-f229-4b4b-a63f-33df297561f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:03.464379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f90bae-f229-4b4b-a63f-33df297561f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:03.475986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f90bae-f229-4b4b-a63f-33df297561f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-20f90bae-f229-4b4b-a63f-33df297561f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:03.480613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:03.481523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:03.483386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:03.483386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:03.483386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:03.483386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:03.483386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:03.483386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:03.483386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:03.483386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:03.483386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:03.483386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:03.483386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:03.483386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:03.483386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 250/491] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:03.484309 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f90bae-f229-4b4b-a63f-33df297561f2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:03.494033 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c80bbb-caa0-4e31-8d63-23c21a4a327e None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:03.531574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73f5b5-a26c-4fe0-86d8-5f2d525d48d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:03.532268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73f5b5-a26c-4fe0-86d8-5f2d525d48d5 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:03.532548 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73f5b5-a26c-4fe0-86d8-5f2d525d48d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:03.532974 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73f5b5-a26c-4fe0-86d8-5f2d525d48d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:03.533571 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73f5b5-a26c-4fe0-86d8-5f2d525d48d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:03.534113 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73f5b5-a26c-4fe0-86d8-5f2d525d48d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:03.535365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73f5b5-a26c-4fe0-86d8-5f2d525d48d5 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:03.550072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 242/492] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:03.550072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029322a-0903-424c-9731-1abb01e9a3f0 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:03.550072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029322a-0903-424c-9731-1abb01e9a3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:03.550072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029322a-0903-424c-9731-1abb01e9a3f0 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:03.551059 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 243/493] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:03.551612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029322a-0903-424c-9731-1abb01e9a3f0 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:03.592465 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f90bae-f229-4b4b-a63f-33df297561f2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:03.630262 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabc081-f7db-403f-8f22-afa78e1c5bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:03.630946 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabc081-f7db-403f-8f22-afa78e1c5bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:03.631219 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabc081-f7db-403f-8f22-afa78e1c5bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:03.631510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabc081-f7db-403f-8f22-afa78e1c5bd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:03.632135 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabc081-f7db-403f-8f22-afa78e1c5bd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:03.632457 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabc081-f7db-403f-8f22-afa78e1c5bd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:03.633912 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabc081-f7db-403f-8f22-afa78e1c5bd3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:03.645974 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 251/494] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:03.646674 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabc081-f7db-403f-8f22-afa78e1c5bd3 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:03.666293 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:03.668636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029322a-0903-424c-9731-1abb01e9a3f0 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:03.678271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:03.708375 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45844e-2f50-4a33-96a5-e59dd77b1f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:03.708987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45844e-2f50-4a33-96a5-e59dd77b1f72 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:03.709263 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45844e-2f50-4a33-96a5-e59dd77b1f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:03.709534 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45844e-2f50-4a33-96a5-e59dd77b1f72 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:03.710198 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45844e-2f50-4a33-96a5-e59dd77b1f72 None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:03.710498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45844e-2f50-4a33-96a5-e59dd77b1f72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:03.711911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45844e-2f50-4a33-96a5-e59dd77b1f72 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:03.720878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 244/495] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:03.722111 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60803a1-4288-4fe0-bd17-c01383479341 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:03.722440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60803a1-4288-4fe0-bd17-c01383479341 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:03.722718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60803a1-4288-4fe0-bd17-c01383479341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:03.739984 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-c60803a1-4288-4fe0-bd17-c01383479341 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:03.741161 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c60803a1-4288-4fe0-bd17-c01383479341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:03.755957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabc081-f7db-403f-8f22-afa78e1c5bd3 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:03.789675 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf11f8-fcb2-40d7-aace-830b87f44b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:03.790317 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf11f8-fcb2-40d7-aace-830b87f44b49 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:03.790617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf11f8-fcb2-40d7-aace-830b87f44b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:03.790885 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf11f8-fcb2-40d7-aace-830b87f44b49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:03.802163 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf11f8-fcb2-40d7-aace-830b87f44b49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:03.803655 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf11f8-fcb2-40d7-aace-830b87f44b49 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:03.808204 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 245/496] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:03.809042 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60803a1-4288-4fe0-bd17-c01383479341 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-35bf11f8-fcb2-40d7-aace-830b87f44b49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:03.829034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:03.829978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:03.830829 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:03.830829 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:03.830829 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:03.830829 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 252/497] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:03.831541 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ed505-7b17-4cbb-b40c-4087bd40878a None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:03.831541 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ed505-7b17-4cbb-b40c-4087bd40878a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:03.831829 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ed505-7b17-4cbb-b40c-4087bd40878a None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:03.833190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 253/498] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:03.833730 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ed505-7b17-4cbb-b40c-4087bd40878a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:03.913727 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:03.924041 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:03.928495 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60803a1-4288-4fe0-bd17-c01383479341 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:03.959134 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ed505-7b17-4cbb-b40c-4087bd40878a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:03.965598 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736a31a-8b1f-431d-aa97-99d38d1ca06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:03.966321 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736a31a-8b1f-431d-aa97-99d38d1ca06d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:03.966683 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736a31a-8b1f-431d-aa97-99d38d1ca06d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:03.966683 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736a31a-8b1f-431d-aa97-99d38d1ca06d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:03.974420 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736a31a-8b1f-431d-aa97-99d38d1ca06d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:03.975876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736a31a-8b1f-431d-aa97-99d38d1ca06d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-e736a31a-8b1f-431d-aa97-99d38d1ca06d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:03.989048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:03.990538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:03.991522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:03.991522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:03.991522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:03.991522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:03.991522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:03.991522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:03.991522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:03.991522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:03.991522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:00:03.991522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:03.991522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 246/499] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 23 06:00:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:03.991522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736a31a-8b1f-431d-aa97-99d38d1ca06d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:04.003717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bca8df-6567-45b9-b69a-8a9e35ae76af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:04.004350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bca8df-6567-45b9-b69a-8a9e35ae76af None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:04.004628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bca8df-6567-45b9-b69a-8a9e35ae76af None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:04.004894 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bca8df-6567-45b9-b69a-8a9e35ae76af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:04.017216 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bca8df-6567-45b9-b69a-8a9e35ae76af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:04.018706 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bca8df-6567-45b9-b69a-8a9e35ae76af None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-d1bca8df-6567-45b9-b69a-8a9e35ae76af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:04.032844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:04.033767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:04.034632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:04.034632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:04.034632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:04.038861 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 254/500] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:04.040113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e309a-7db8-4f3e-9d80-c9dac6f3fe14 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:04.040400 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e309a-7db8-4f3e-9d80-c9dac6f3fe14 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:04.040678 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e309a-7db8-4f3e-9d80-c9dac6f3fe14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:04.058605 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-9b1e309a-7db8-4f3e-9d80-c9dac6f3fe14 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:04.060354 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9b1e309a-7db8-4f3e-9d80-c9dac6f3fe14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:04.107380 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736a31a-8b1f-431d-aa97-99d38d1ca06d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:04.130214 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 255/501] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:04.131718 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e309a-7db8-4f3e-9d80-c9dac6f3fe14 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:04.150312 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9838b-aee9-43ab-af86-44b311313053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:04.150963 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9838b-aee9-43ab-af86-44b311313053 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:04.151240 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9838b-aee9-43ab-af86-44b311313053 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:04.151514 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9838b-aee9-43ab-af86-44b311313053 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:04.152156 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9838b-aee9-43ab-af86-44b311313053 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:04.152461 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9838b-aee9-43ab-af86-44b311313053 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:04.153893 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9838b-aee9-43ab-af86-44b311313053 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:04.174287 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 247/502] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:04.174287 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9838b-aee9-43ab-af86-44b311313053 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:04.195851 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:04.196823 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:04.249658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e309a-7db8-4f3e-9d80-c9dac6f3fe14 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:04.285387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6ff09-628f-44b7-b0ab-163240a23d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:04.286005 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6ff09-628f-44b7-b0ab-163240a23d6c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:04.286281 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6ff09-628f-44b7-b0ab-163240a23d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:04.286573 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6ff09-628f-44b7-b0ab-163240a23d6c None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:04.287310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6ff09-628f-44b7-b0ab-163240a23d6c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:04.287613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6ff09-628f-44b7-b0ab-163240a23d6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:04.289051 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6ff09-628f-44b7-b0ab-163240a23d6c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:04.301962 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9838b-aee9-43ab-af86-44b311313053 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:04.327227 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 256/503] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 23 06:00:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:04.328083 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6ff09-628f-44b7-b0ab-163240a23d6c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:04.343189 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9548dec-af05-4e41-be31-c8f6f6b4a458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:04.343996 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9548dec-af05-4e41-be31-c8f6f6b4a458 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:04.344412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9548dec-af05-4e41-be31-c8f6f6b4a458 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:04.344841 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9548dec-af05-4e41-be31-c8f6f6b4a458 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:04.345623 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9548dec-af05-4e41-be31-c8f6f6b4a458 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:04.345922 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9548dec-af05-4e41-be31-c8f6f6b4a458 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:04.347310 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9548dec-af05-4e41-be31-c8f6f6b4a458 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:04.359707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 248/504] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:04.359707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9548dec-af05-4e41-be31-c8f6f6b4a458 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:04.438738 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6ff09-628f-44b7-b0ab-163240a23d6c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:04.444139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:04.449528 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:04.466268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9548dec-af05-4e41-be31-c8f6f6b4a458 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:04.473753 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57aa83-8d1c-4c0e-8275-8979670121cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:04.474374 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57aa83-8d1c-4c0e-8275-8979670121cc None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:04.474680 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57aa83-8d1c-4c0e-8275-8979670121cc None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:04.474957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57aa83-8d1c-4c0e-8275-8979670121cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:04.486383 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57aa83-8d1c-4c0e-8275-8979670121cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:04.487872 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57aa83-8d1c-4c0e-8275-8979670121cc None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:04.501093 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2bf90-72f2-40cf-9ede-0892aa622ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:04.501718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2bf90-72f2-40cf-9ede-0892aa622ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:04.501998 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2bf90-72f2-40cf-9ede-0892aa622ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:04.502271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2bf90-72f2-40cf-9ede-0892aa622ec9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-fc57aa83-8d1c-4c0e-8275-8979670121cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:04.504112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:04.505064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:04.506273 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:04.506273 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:04.506273 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:04.506273 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 257/505] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:04.506273 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57aa83-8d1c-4c0e-8275-8979670121cc None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:04.511591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2bf90-72f2-40cf-9ede-0892aa622ec9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:04.513143 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2bf90-72f2-40cf-9ede-0892aa622ec9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-87b2bf90-72f2-40cf-9ede-0892aa622ec9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:04.529578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:04.530488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:04.531408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:04.531408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:04.531408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:04.531408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:04.531408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:04.531408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:04.531408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:04.531408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:04.531408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 06:00:04.531408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:04.531408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 249/506] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:00:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:04.532493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2bf90-72f2-40cf-9ede-0892aa622ec9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:04.620313 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57aa83-8d1c-4c0e-8275-8979670121cc None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:04.641465 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2bf90-72f2-40cf-9ede-0892aa622ec9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:04.655353 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe79c03-cad8-4020-b7de-30c824710539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:04.655964 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe79c03-cad8-4020-b7de-30c824710539 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:04.656245 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe79c03-cad8-4020-b7de-30c824710539 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:04.656519 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe79c03-cad8-4020-b7de-30c824710539 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:04.675267 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf4a89-c340-4930-8d6d-06a71c46d3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:04.675886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf4a89-c340-4930-8d6d-06a71c46d3ad None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:04.676160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf4a89-c340-4930-8d6d-06a71c46d3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:04.676429 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf4a89-c340-4930-8d6d-06a71c46d3ad None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:04.683747 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf4a89-c340-4930-8d6d-06a71c46d3ad None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:04.685567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf4a89-c340-4930-8d6d-06a71c46d3ad None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:04.686845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:04.687349 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-ebaf4a89-c340-4930-8d6d-06a71c46d3ad None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:04.698436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:04.699529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:04.700436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:04.700436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:04.700436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:04.700436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:04.700436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:04.700436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:04.700436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:04.700436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:04.700436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 06:00:04.700436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:04.700436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 250/507] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:00:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:04.700436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf4a89-c340-4930-8d6d-06a71c46d3ad None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:04.730249 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe79c03-cad8-4020-b7de-30c824710539 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:04.731829 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe79c03-cad8-4020-b7de-30c824710539 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:04.807022 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf4a89-c340-4930-8d6d-06a71c46d3ad None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:04.834177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 258/508] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:00:04.835062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe79c03-cad8-4020-b7de-30c824710539 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:04.840762 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a8e34-34f4-4b3a-9c39-6768b6fa9421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:04.841402 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a8e34-34f4-4b3a-9c39-6768b6fa9421 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:04.841679 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a8e34-34f4-4b3a-9c39-6768b6fa9421 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:04.841944 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a8e34-34f4-4b3a-9c39-6768b6fa9421 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:04.852038 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a8e34-34f4-4b3a-9c39-6768b6fa9421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-798a8e34-34f4-4b3a-9c39-6768b6fa9421 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:04.857881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:04.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:04.859685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:04.859685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:04.859685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:04.859685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:04.859685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:04.859685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:04.859685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:04.859685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:04.859685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:04.859685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:04.859685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:04.859685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:04.859685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 251/509] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:04.860335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a8e34-34f4-4b3a-9c39-6768b6fa9421 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:04.899878 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:04.928801 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:04.941011 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe79c03-cad8-4020-b7de-30c824710539 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:04.975802 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a8e34-34f4-4b3a-9c39-6768b6fa9421 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:04.977599 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25951500-a5fb-4e48-937e-7b8f569e0ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:04.978199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25951500-a5fb-4e48-937e-7b8f569e0ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:04.978474 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25951500-a5fb-4e48-937e-7b8f569e0ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:04.978771 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25951500-a5fb-4e48-937e-7b8f569e0ce8 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:04.979412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25951500-a5fb-4e48-937e-7b8f569e0ce8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:04.979711 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25951500-a5fb-4e48-937e-7b8f569e0ce8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:04.981100 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25951500-a5fb-4e48-937e-7b8f569e0ce8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:04.993314 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 259/510] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:04.994181 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25951500-a5fb-4e48-937e-7b8f569e0ce8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:05.010664 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38497be5-5509-4dfc-b5b9-2795343a84a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:05.011273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38497be5-5509-4dfc-b5b9-2795343a84a3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:05.011551 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38497be5-5509-4dfc-b5b9-2795343a84a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:05.011821 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38497be5-5509-4dfc-b5b9-2795343a84a3 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:05.012547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38497be5-5509-4dfc-b5b9-2795343a84a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:05.012852 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38497be5-5509-4dfc-b5b9-2795343a84a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:05.014382 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38497be5-5509-4dfc-b5b9-2795343a84a3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:05.051523 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 252/511] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 23 06:00:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:05.052369 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38497be5-5509-4dfc-b5b9-2795343a84a3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:05.097841 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25951500-a5fb-4e48-937e-7b8f569e0ce8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:05.120251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:05.134053 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110473a-846f-439d-9dc9-82e9bed0c15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:05.134728 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110473a-846f-439d-9dc9-82e9bed0c15f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:05.134932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110473a-846f-439d-9dc9-82e9bed0c15f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:05.135194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110473a-846f-439d-9dc9-82e9bed0c15f None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:05.135921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110473a-846f-439d-9dc9-82e9bed0c15f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:05.136216 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110473a-846f-439d-9dc9-82e9bed0c15f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:05.137708 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110473a-846f-439d-9dc9-82e9bed0c15f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:05.152197 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38497be5-5509-4dfc-b5b9-2795343a84a3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:05.157028 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 260/512] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 23 06:00:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:05.157855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110473a-846f-439d-9dc9-82e9bed0c15f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:05.169682 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:05.188418 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4dd940-253b-4053-83a7-6ab0bac34f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:05.189049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4dd940-253b-4053-83a7-6ab0bac34f67 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:05.189323 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4dd940-253b-4053-83a7-6ab0bac34f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:05.189596 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4dd940-253b-4053-83a7-6ab0bac34f67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:05.199762 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4dd940-253b-4053-83a7-6ab0bac34f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:05.201870 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4dd940-253b-4053-83a7-6ab0bac34f67 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-5b4dd940-253b-4053-83a7-6ab0bac34f67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:05.216299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:05.217350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:05.218388 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:05.218388 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:05.218388 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:05.218388 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 253/513] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:05.218388 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4dd940-253b-4053-83a7-6ab0bac34f67 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:05.261217 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110473a-846f-439d-9dc9-82e9bed0c15f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:05.296587 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfe8b6-a343-4f4f-9efd-611256d7f1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:05.297186 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfe8b6-a343-4f4f-9efd-611256d7f1e4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:05.297472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfe8b6-a343-4f4f-9efd-611256d7f1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:05.297738 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfe8b6-a343-4f4f-9efd-611256d7f1e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:05.298422 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cfe8b6-a343-4f4f-9efd-611256d7f1e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:05.298748 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cfe8b6-a343-4f4f-9efd-611256d7f1e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:05.300124 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cfe8b6-a343-4f4f-9efd-611256d7f1e4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:05.308362 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 261/514] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:05.309176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfe8b6-a343-4f4f-9efd-611256d7f1e4 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:05.320250 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4dd940-253b-4053-83a7-6ab0bac34f67 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:05.353128 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a25f1-5646-4337-ba59-4f72679426cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:05.353725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a25f1-5646-4337-ba59-4f72679426cc None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:05.353999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a25f1-5646-4337-ba59-4f72679426cc None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:05.354260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a25f1-5646-4337-ba59-4f72679426cc None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:05.354951 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a25f1-5646-4337-ba59-4f72679426cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:05.356002 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:05.357468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a25f1-5646-4337-ba59-4f72679426cc None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:05.380318 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 254/515] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 06:00:05 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:00:05.381510 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a25f1-5646-4337-ba59-4f72679426cc None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:05.410738 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:05.446443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfe8b6-a343-4f4f-9efd-611256d7f1e4 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:05.486237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e668f6-c6b5-4efd-94f7-73a9ec6c2f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:05.486804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e668f6-c6b5-4efd-94f7-73a9ec6c2f31 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:05.487087 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e668f6-c6b5-4efd-94f7-73a9ec6c2f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:05.487363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e668f6-c6b5-4efd-94f7-73a9ec6c2f31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:05.505825 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a25f1-5646-4337-ba59-4f72679426cc None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:05.545118 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7c088-e89e-47f5-8cb7-6ae44c6a0fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:05.545739 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7c088-e89e-47f5-8cb7-6ae44c6a0fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:05.546019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7c088-e89e-47f5-8cb7-6ae44c6a0fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:05.546715 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7c088-e89e-47f5-8cb7-6ae44c6a0fe8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:05.569073 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e668f6-c6b5-4efd-94f7-73a9ec6c2f31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:05.571359 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e668f6-c6b5-4efd-94f7-73a9ec6c2f31 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:05.623218 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:05.632554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7c088-e89e-47f5-8cb7-6ae44c6a0fe8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:05.634648 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7c088-e89e-47f5-8cb7-6ae44c6a0fe8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:05.667931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:05.684479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 262/516] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:00:05.685580 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e668f6-c6b5-4efd-94f7-73a9ec6c2f31 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:05.758407 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 255/517] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:00:05.759508 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7c088-e89e-47f5-8cb7-6ae44c6a0fe8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:05.803471 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e668f6-c6b5-4efd-94f7-73a9ec6c2f31 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:05.840957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f43311-95a3-4d9e-98f4-3860bafade36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:05.842234 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f43311-95a3-4d9e-98f4-3860bafade36 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:05.842543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f43311-95a3-4d9e-98f4-3860bafade36 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:05.842809 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f43311-95a3-4d9e-98f4-3860bafade36 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:05.843514 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f43311-95a3-4d9e-98f4-3860bafade36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:05.843747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f43311-95a3-4d9e-98f4-3860bafade36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:05.845115 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f43311-95a3-4d9e-98f4-3860bafade36 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:05.861953 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 263/518] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:05.861953 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f43311-95a3-4d9e-98f4-3860bafade36 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:05.865371 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:05.865612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7c088-e89e-47f5-8cb7-6ae44c6a0fe8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:05.895239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:05.907684 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e8c4d-7e85-486a-a3f5-c34b660203fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:05.908320 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e8c4d-7e85-486a-a3f5-c34b660203fe None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:05.908591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e8c4d-7e85-486a-a3f5-c34b660203fe None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:05.908862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e8c4d-7e85-486a-a3f5-c34b660203fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:05.919709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e8c4d-7e85-486a-a3f5-c34b660203fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:05.921146 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e8c4d-7e85-486a-a3f5-c34b660203fe None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-037e8c4d-7e85-486a-a3f5-c34b660203fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:05.936174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:05.937094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:05.937949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:05.937949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:05.937949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:05.937949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 256/519] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:05.938621 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e8c4d-7e85-486a-a3f5-c34b660203fe None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:06.000015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f43311-95a3-4d9e-98f4-3860bafade36 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:06.035546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc729d-2aed-4b3e-9861-6e5101e9b3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:06.036095 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc729d-2aed-4b3e-9861-6e5101e9b3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:06.036375 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc729d-2aed-4b3e-9861-6e5101e9b3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:06.036646 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc729d-2aed-4b3e-9861-6e5101e9b3e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:06.048641 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc729d-2aed-4b3e-9861-6e5101e9b3e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:06.050113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc729d-2aed-4b3e-9861-6e5101e9b3e3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:06.051392 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e8c4d-7e85-486a-a3f5-c34b660203fe None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-f9cc729d-2aed-4b3e-9861-6e5101e9b3e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:06.065326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:06.066265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:06.067172 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:06.067172 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:06.067172 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:06.067172 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 264/520] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:06.067814 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc729d-2aed-4b3e-9861-6e5101e9b3e3 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:06.095081 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4670d7-231f-4cfc-8826-9aa8281b9a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:06.096159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4670d7-231f-4cfc-8826-9aa8281b9a01 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:06.096602 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4670d7-231f-4cfc-8826-9aa8281b9a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:06.097043 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4670d7-231f-4cfc-8826-9aa8281b9a01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:06.108613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4670d7-231f-4cfc-8826-9aa8281b9a01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:06.110371 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4670d7-231f-4cfc-8826-9aa8281b9a01 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-ea4670d7-231f-4cfc-8826-9aa8281b9a01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:06.126177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:06.127273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:06.128279 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:06.128279 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:06.128279 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:06.128804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:06.134953 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 257/521] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:06.136009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4670d7-231f-4cfc-8826-9aa8281b9a01 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:06.165139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:06.183469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc729d-2aed-4b3e-9861-6e5101e9b3e3 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:06.222635 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d1b18-1245-4243-9198-262cfaaabeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:06.223261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d1b18-1245-4243-9198-262cfaaabeca None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:06.223543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d1b18-1245-4243-9198-262cfaaabeca None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:06.223815 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d1b18-1245-4243-9198-262cfaaabeca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:06.232579 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d1b18-1245-4243-9198-262cfaaabeca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:06.234246 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d1b18-1245-4243-9198-262cfaaabeca None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:06.247059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-c23d1b18-1245-4243-9198-262cfaaabeca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:06.247059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:06.247059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:06.247059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:06.247059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:06.247059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:06.247059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:06.247059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:06.247059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:06.247059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:06.247059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:06.247059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:06.247059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:06.247059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:06.247059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:06.248195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:06.249089 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:06.249089 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:06.249089 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:06.249089 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:06.249089 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:06.249089 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:06.249089 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:06.249089 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:06.249516 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4670d7-231f-4cfc-8826-9aa8281b9a01 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:06.253450 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 265/522] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:06.254347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d1b18-1245-4243-9198-262cfaaabeca None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:06.286389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9297589-fac4-43cd-bbae-e180882adee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:06.287020 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9297589-fac4-43cd-bbae-e180882adee8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:06.287207 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9297589-fac4-43cd-bbae-e180882adee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:06.287478 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9297589-fac4-43cd-bbae-e180882adee8 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:06.288122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9297589-fac4-43cd-bbae-e180882adee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:06.288427 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9297589-fac4-43cd-bbae-e180882adee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:06.289845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9297589-fac4-43cd-bbae-e180882adee8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:06.302119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 258/523] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:06.307373 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9297589-fac4-43cd-bbae-e180882adee8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:06.366264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d1b18-1245-4243-9198-262cfaaabeca None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:06.383348 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:06.406556 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:06.406860 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093f993-bf80-4a20-8576-6a8cbc04d436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:06.407557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093f993-bf80-4a20-8576-6a8cbc04d436 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:06.407747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093f993-bf80-4a20-8576-6a8cbc04d436 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:06.408104 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093f993-bf80-4a20-8576-6a8cbc04d436 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:06.408686 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093f993-bf80-4a20-8576-6a8cbc04d436 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:06.408982 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093f993-bf80-4a20-8576-6a8cbc04d436 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:06.410454 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093f993-bf80-4a20-8576-6a8cbc04d436 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:06.417806 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9297589-fac4-43cd-bbae-e180882adee8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:06.428928 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 266/524] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:06.428928 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093f993-bf80-4a20-8576-6a8cbc04d436 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:06.457823 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18722f56-19e3-4479-b815-c7573c1bcaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:06.458575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18722f56-19e3-4479-b815-c7573c1bcaf4 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:06.458921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18722f56-19e3-4479-b815-c7573c1bcaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:06.459190 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18722f56-19e3-4479-b815-c7573c1bcaf4 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:06.459827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18722f56-19e3-4479-b815-c7573c1bcaf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:06.460145 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18722f56-19e3-4479-b815-c7573c1bcaf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:06.461488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18722f56-19e3-4479-b815-c7573c1bcaf4 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:06.473003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 259/525] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:06.473827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18722f56-19e3-4479-b815-c7573c1bcaf4 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:06.539752 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093f993-bf80-4a20-8576-6a8cbc04d436 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:06.573829 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27f959-5dc3-49a9-b93d-fde05b08a25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:06.574562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27f959-5dc3-49a9-b93d-fde05b08a25b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:06.574849 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27f959-5dc3-49a9-b93d-fde05b08a25b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:06.575128 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27f959-5dc3-49a9-b93d-fde05b08a25b None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:06.576621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27f959-5dc3-49a9-b93d-fde05b08a25b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:06.576943 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27f959-5dc3-49a9-b93d-fde05b08a25b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:06.578386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27f959-5dc3-49a9-b93d-fde05b08a25b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:06.583807 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18722f56-19e3-4479-b815-c7573c1bcaf4 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:06.602039 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 267/526] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:06.602039 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27f959-5dc3-49a9-b93d-fde05b08a25b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:06.625334 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d59aa-7bb7-4bd2-bd2c-f4de5b5ec122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:06.626212 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d59aa-7bb7-4bd2-bd2c-f4de5b5ec122 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:06.626489 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d59aa-7bb7-4bd2-bd2c-f4de5b5ec122 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:06.626783 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d59aa-7bb7-4bd2-bd2c-f4de5b5ec122 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:06.634669 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d59aa-7bb7-4bd2-bd2c-f4de5b5ec122 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:06.636102 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d59aa-7bb7-4bd2-bd2c-f4de5b5ec122 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:06.640327 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-020d59aa-7bb7-4bd2-bd2c-f4de5b5ec122 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:06.652044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:06.653063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:06.654079 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:06.654079 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:06.654079 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:06.654079 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:06.654079 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:06.654079 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:06.654079 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:06.654079 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:06.654079 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 06:00:06.654079 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:06.654079 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 260/527] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:06.654668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d59aa-7bb7-4bd2-bd2c-f4de5b5ec122 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:06.667114 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:06.719716 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27f959-5dc3-49a9-b93d-fde05b08a25b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:06.757307 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cafcf9-eac7-46f6-a0c8-80106e9febc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:06.758056 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cafcf9-eac7-46f6-a0c8-80106e9febc3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:06.758333 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cafcf9-eac7-46f6-a0c8-80106e9febc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:06.758636 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cafcf9-eac7-46f6-a0c8-80106e9febc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:06.766243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cafcf9-eac7-46f6-a0c8-80106e9febc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:06.767748 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cafcf9-eac7-46f6-a0c8-80106e9febc3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:06.771300 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d59aa-7bb7-4bd2-bd2c-f4de5b5ec122 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:06.780602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-d2cafcf9-eac7-46f6-a0c8-80106e9febc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:06.780602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:06.780602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:06.780602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:06.780602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:06.780602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:06.780602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:06.780602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:06.780602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:06.780602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:06.780602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:06.780602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:06.780602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:06.780602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:06.780602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:06.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:06.782417 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:06.782417 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:06.782417 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:06.782417 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:06.782417 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:06.782417 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:06.782417 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:06.782417 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:06.782417 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 268/528] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:06.783873 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cafcf9-eac7-46f6-a0c8-80106e9febc3 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:06.818905 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f4960-79c3-49fb-ade5-3de6864da42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:06.819525 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f4960-79c3-49fb-ade5-3de6864da42a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:06.819753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f4960-79c3-49fb-ade5-3de6864da42a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:06.820031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f4960-79c3-49fb-ade5-3de6864da42a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:06.829310 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f4960-79c3-49fb-ade5-3de6864da42a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:06.831141 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f4960-79c3-49fb-ade5-3de6864da42a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:06.844627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-b36f4960-79c3-49fb-ade5-3de6864da42a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:06.844627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:06.844627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:06.844627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:06.844627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:06.844627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:06.844627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:06.844627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:06.844627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:06.844627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:06.844627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:06.844627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:06.844627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:06.844627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:06.844627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:06.845636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:06.846539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:06.846539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:06.846539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:06.846539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:06.846539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:06.846539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:06.846539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:06.846539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:06.846539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 261/529] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:06.847131 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f4960-79c3-49fb-ade5-3de6864da42a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:06.895387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:06.905171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cafcf9-eac7-46f6-a0c8-80106e9febc3 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:06.930621 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:06.948332 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d70e0-c196-4b89-b46f-61ab19237746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:06.948935 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d70e0-c196-4b89-b46f-61ab19237746 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:06.949217 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d70e0-c196-4b89-b46f-61ab19237746 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:06.949486 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d70e0-c196-4b89-b46f-61ab19237746 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:06.950242 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d70e0-c196-4b89-b46f-61ab19237746 None admin] RBAC: Adding query filter params (role.id=3209a3d044b84a0ebd7cf1edf2da7120, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=91267a049343452eb208205ee3ea27c4) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:06.950662 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d70e0-c196-4b89-b46f-61ab19237746 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:06.952129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d70e0-c196-4b89-b46f-61ab19237746 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:06.966757 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 269/530] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:06 2022] GET /identity/v3/role_assignments?user.id=91267a049343452eb208205ee3ea27c4&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=3209a3d044b84a0ebd7cf1edf2da7120 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:06.966757 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d70e0-c196-4b89-b46f-61ab19237746 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:06.988862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f4960-79c3-49fb-ade5-3de6864da42a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:07.034817 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7765a-ea82-4792-97be-2e35c5067817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:07.035631 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7765a-ea82-4792-97be-2e35c5067817 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:07.035990 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7765a-ea82-4792-97be-2e35c5067817 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:07.036334 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7765a-ea82-4792-97be-2e35c5067817 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:07.037177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7765a-ea82-4792-97be-2e35c5067817 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:07.037556 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7765a-ea82-4792-97be-2e35c5067817 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:07.039269 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7765a-ea82-4792-97be-2e35c5067817 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:07.066341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 262/531] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Aug 23 06:00:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:07.067582 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7765a-ea82-4792-97be-2e35c5067817 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:07.103463 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d70e0-c196-4b89-b46f-61ab19237746 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:07.159657 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ace1c-3f47-44b8-b416-4eca25fbb7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:07.160292 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ace1c-3f47-44b8-b416-4eca25fbb7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:07.160571 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ace1c-3f47-44b8-b416-4eca25fbb7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:07.160842 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ace1c-3f47-44b8-b416-4eca25fbb7f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:07.161983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ace1c-3f47-44b8-b416-4eca25fbb7f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:07.162294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ace1c-3f47-44b8-b416-4eca25fbb7f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:07.164152 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ace1c-3f47-44b8-b416-4eca25fbb7f8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:07.186544 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 270/532] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:07.186544 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ace1c-3f47-44b8-b416-4eca25fbb7f8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:07.196829 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7765a-ea82-4792-97be-2e35c5067817 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:07.215066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:07.231753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:07.242753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967d7d8-4679-4e6a-b4ec-ffc116a913fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:07.243709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967d7d8-4679-4e6a-b4ec-ffc116a913fe None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:07.244178 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967d7d8-4679-4e6a-b4ec-ffc116a913fe None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:07.244651 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967d7d8-4679-4e6a-b4ec-ffc116a913fe None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:07.245534 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967d7d8-4679-4e6a-b4ec-ffc116a913fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:07.246031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967d7d8-4679-4e6a-b4ec-ffc116a913fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:07.247905 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967d7d8-4679-4e6a-b4ec-ffc116a913fe None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:07.267165 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 263/533] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:07.268423 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967d7d8-4679-4e6a-b4ec-ffc116a913fe None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:07.331911 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ace1c-3f47-44b8-b416-4eca25fbb7f8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:07.371133 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c7c920-38d3-4011-8c46-58cc0a4b5a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:07.371666 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7c920-38d3-4011-8c46-58cc0a4b5a24 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:07.371946 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7c920-38d3-4011-8c46-58cc0a4b5a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:07.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7c920-38d3-4011-8c46-58cc0a4b5a24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:07.375521 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967d7d8-4679-4e6a-b4ec-ffc116a913fe None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:07.417571 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93771dc-a5e5-4475-98b7-e7db3a347b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:07.418347 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93771dc-a5e5-4475-98b7-e7db3a347b4c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:07.418749 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93771dc-a5e5-4475-98b7-e7db3a347b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:07.419156 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93771dc-a5e5-4475-98b7-e7db3a347b4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:07.420025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93771dc-a5e5-4475-98b7-e7db3a347b4c None admin] RBAC: Adding query filter params (role.id=3209a3d044b84a0ebd7cf1edf2da7120, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=89621d839d1c44798f691db4d49e1278) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:07.420345 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93771dc-a5e5-4475-98b7-e7db3a347b4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:07.421827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93771dc-a5e5-4475-98b7-e7db3a347b4c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:07.427830 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 264/534] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:07 2022] GET /identity/v3/role_assignments?user.id=89621d839d1c44798f691db4d49e1278&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=3209a3d044b84a0ebd7cf1edf2da7120 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:07.439648 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93771dc-a5e5-4475-98b7-e7db3a347b4c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:07.457638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7c920-38d3-4011-8c46-58cc0a4b5a24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:07.459566 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7c920-38d3-4011-8c46-58cc0a4b5a24 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:07.480777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:07.516453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:07.590034 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93771dc-a5e5-4475-98b7-e7db3a347b4c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:07.596919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 271/535] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:00:07.598259 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53f7a3-0be5-4953-84f8-14fa23a31dde None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:07.598557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53f7a3-0be5-4953-84f8-14fa23a31dde None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:07.598830 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53f7a3-0be5-4953-84f8-14fa23a31dde None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:07.600326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 272/536] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:07.602347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53f7a3-0be5-4953-84f8-14fa23a31dde None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:07.631122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5f4cf-c1bb-454d-8c97-5e8226351c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:07.632325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5f4cf-c1bb-454d-8c97-5e8226351c52 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:07.632802 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5f4cf-c1bb-454d-8c97-5e8226351c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:07.633269 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5f4cf-c1bb-454d-8c97-5e8226351c52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:07.634313 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5f4cf-c1bb-454d-8c97-5e8226351c52 None admin] RBAC: Adding query filter params (role.id=3209a3d044b84a0ebd7cf1edf2da7120, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=c741b92684794c338d63913774ab841a) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:07.634932 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5f4cf-c1bb-454d-8c97-5e8226351c52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:07.636769 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5f4cf-c1bb-454d-8c97-5e8226351c52 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:07.644589 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 265/537] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:07 2022] GET /identity/v3/role_assignments?user.id=c741b92684794c338d63913774ab841a&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=3209a3d044b84a0ebd7cf1edf2da7120 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:07.646419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d6eb5-77d1-4f9e-9ef5-7d73841ece26 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:07.647001 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d6eb5-77d1-4f9e-9ef5-7d73841ece26 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:07.647490 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d6eb5-77d1-4f9e-9ef5-7d73841ece26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:07.668086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ff3d6eb5-77d1-4f9e-9ef5-7d73841ece26 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:07.669788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ff3d6eb5-77d1-4f9e-9ef5-7d73841ece26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:07.713272 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53f7a3-0be5-4953-84f8-14fa23a31dde None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:07.732011 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:07.753216 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee13d1-ea8f-4c68-af45-3466bf90ba48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:07.753910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee13d1-ea8f-4c68-af45-3466bf90ba48 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:07.754121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee13d1-ea8f-4c68-af45-3466bf90ba48 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:07.754553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee13d1-ea8f-4c68-af45-3466bf90ba48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:07.754801 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 266/538] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:07.765600 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee13d1-ea8f-4c68-af45-3466bf90ba48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:07.767189 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee13d1-ea8f-4c68-af45-3466bf90ba48 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-c3ee13d1-ea8f-4c68-af45-3466bf90ba48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:07.784063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:07.785212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:07.786209 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:07.786209 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:07.786209 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:07.787908 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 273/539] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:07.793223 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d6eb5-77d1-4f9e-9ef5-7d73841ece26 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:07.874409 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:07.939862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d6eb5-77d1-4f9e-9ef5-7d73841ece26 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:07.994643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db0b93-3910-4ba6-9702-c22fded59fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:07.995344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db0b93-3910-4ba6-9702-c22fded59fcb None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:07.995685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db0b93-3910-4ba6-9702-c22fded59fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:07.996022 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db0b93-3910-4ba6-9702-c22fded59fcb None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:07.996733 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db0b93-3910-4ba6-9702-c22fded59fcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:07.997124 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db0b93-3910-4ba6-9702-c22fded59fcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:07.998674 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db0b93-3910-4ba6-9702-c22fded59fcb None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:08.013476 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 267/540] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:08.019544 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee13d1-ea8f-4c68-af45-3466bf90ba48 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:08.049799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db0b93-3910-4ba6-9702-c22fded59fcb None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:08.159172 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db0b93-3910-4ba6-9702-c22fded59fcb None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:08.162029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:08.197376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172054cf-8d3c-4584-8743-00c8fe1f281b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:08.197985 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172054cf-8d3c-4584-8743-00c8fe1f281b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:08.198243 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172054cf-8d3c-4584-8743-00c8fe1f281b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:08.198454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172054cf-8d3c-4584-8743-00c8fe1f281b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:08.211576 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172054cf-8d3c-4584-8743-00c8fe1f281b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-172054cf-8d3c-4584-8743-00c8fe1f281b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:08.217979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:08.218898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:08.219668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:08.219668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:08.219668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:08.219668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:08.219668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:08.219668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:08.219668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:08.219668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:08.219668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:08.219668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:08.219668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:08.219668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:08.219668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 268/541] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:08.224758 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172054cf-8d3c-4584-8743-00c8fe1f281b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:08.235038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee13d1-ea8f-4c68-af45-3466bf90ba48 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:08.305242 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236434e-872b-4210-954c-626463947c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:08.306185 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236434e-872b-4210-954c-626463947c39 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:08.306471 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236434e-872b-4210-954c-626463947c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:08.306769 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236434e-872b-4210-954c-626463947c39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:08.315357 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8236434e-872b-4210-954c-626463947c39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:08.318750 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8236434e-872b-4210-954c-626463947c39 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:08.329705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-8236434e-872b-4210-954c-626463947c39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:08.329705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:08.329705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:08.329705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:08.329705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:08.329705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:08.329705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:08.329705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:08.329705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:08.329705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:08.329705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:08.329705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:08.329705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:08.329705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:08.329705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:08.330787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:08.331802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:08.331802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:08.331802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:08.331802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:08.331802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:08.331802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:08.331802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:08.331802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:08.333824 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 274/542] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:08.343673 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236434e-872b-4210-954c-626463947c39 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:08.378544 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:08.389454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172054cf-8d3c-4584-8743-00c8fe1f281b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:08.427393 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35c7d5-9ac7-42d6-b007-6fdd563d7e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:08.428285 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35c7d5-9ac7-42d6-b007-6fdd563d7e37 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:08.429009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35c7d5-9ac7-42d6-b007-6fdd563d7e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:08.429446 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35c7d5-9ac7-42d6-b007-6fdd563d7e37 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:08.430277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35c7d5-9ac7-42d6-b007-6fdd563d7e37 None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:08.431011 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35c7d5-9ac7-42d6-b007-6fdd563d7e37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:08.432894 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35c7d5-9ac7-42d6-b007-6fdd563d7e37 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:08.443442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 269/543] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:08.449109 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35c7d5-9ac7-42d6-b007-6fdd563d7e37 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:08.506892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:08.546449 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236434e-872b-4210-954c-626463947c39 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:08.558817 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35c7d5-9ac7-42d6-b007-6fdd563d7e37 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:08.607942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ede28-ce57-4e4f-97d3-7288a90be1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:08.608967 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ede28-ce57-4e4f-97d3-7288a90be1d7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:08.609434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ede28-ce57-4e4f-97d3-7288a90be1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:08.609900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ede28-ce57-4e4f-97d3-7288a90be1d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:08.623028 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ede28-ce57-4e4f-97d3-7288a90be1d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:08.625118 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ede28-ce57-4e4f-97d3-7288a90be1d7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-572ede28-ce57-4e4f-97d3-7288a90be1d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:08.643122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:08.644064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:08.644979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:08.644979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:08.644979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:08.647164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 270/544] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:08.651463 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326b538-71a5-4568-bfa1-df424b86d49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:08.652274 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326b538-71a5-4568-bfa1-df424b86d49f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:08.652769 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326b538-71a5-4568-bfa1-df424b86d49f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:08.652883 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326b538-71a5-4568-bfa1-df424b86d49f None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:08.653584 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326b538-71a5-4568-bfa1-df424b86d49f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:08.653888 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326b538-71a5-4568-bfa1-df424b86d49f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:08.654667 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ede28-ce57-4e4f-97d3-7288a90be1d7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:08.655784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326b538-71a5-4568-bfa1-df424b86d49f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:08.681503 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 275/545] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:08.687351 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326b538-71a5-4568-bfa1-df424b86d49f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:08.743200 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:08.761908 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ede28-ce57-4e4f-97d3-7288a90be1d7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:08.799949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7043be0-8bf8-4b04-9d9d-f96b80d0996c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:08.800942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7043be0-8bf8-4b04-9d9d-f96b80d0996c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:08.801422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7043be0-8bf8-4b04-9d9d-f96b80d0996c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:08.801881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7043be0-8bf8-4b04-9d9d-f96b80d0996c None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:08.802828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7043be0-8bf8-4b04-9d9d-f96b80d0996c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:08.803321 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7043be0-8bf8-4b04-9d9d-f96b80d0996c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:08.804870 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7043be0-8bf8-4b04-9d9d-f96b80d0996c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:08.818162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 271/546] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:08.823918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7043be0-8bf8-4b04-9d9d-f96b80d0996c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:08.888531 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:08.935608 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7043be0-8bf8-4b04-9d9d-f96b80d0996c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:08.939524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326b538-71a5-4568-bfa1-df424b86d49f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:08.999159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f85790-1561-417e-a1e7-018219f1d22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:09.000237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f85790-1561-417e-a1e7-018219f1d22c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:09.000752 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f85790-1561-417e-a1e7-018219f1d22c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:09.001232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f85790-1561-417e-a1e7-018219f1d22c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:09.009474 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f85790-1561-417e-a1e7-018219f1d22c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:09.011495 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f85790-1561-417e-a1e7-018219f1d22c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:09.013601 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-39f85790-1561-417e-a1e7-018219f1d22c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:09.026314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:09.027249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:09.029271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:09.029271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:09.029271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:09.029271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:09.029271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:09.029271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:09.029271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:09.029271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:09.029271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:00:09.029271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:09.031935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 272/547] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:00:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:09.036622 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087c9d3-0a74-4469-83a4-555d2a4e770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:09.036622 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087c9d3-0a74-4469-83a4-555d2a4e770c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:09.036622 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087c9d3-0a74-4469-83a4-555d2a4e770c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:09.036622 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087c9d3-0a74-4469-83a4-555d2a4e770c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:09.037028 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087c9d3-0a74-4469-83a4-555d2a4e770c None admin] RBAC: Adding query filter params (role.id=3209a3d044b84a0ebd7cf1edf2da7120, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=dae9eff3bbf04496af2300a24c630eb5) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:09.037028 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087c9d3-0a74-4469-83a4-555d2a4e770c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:09.037028 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087c9d3-0a74-4469-83a4-555d2a4e770c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:09.038445 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f85790-1561-417e-a1e7-018219f1d22c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:09.049686 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 276/548] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:08 2022] GET /identity/v3/role_assignments?user.id=dae9eff3bbf04496af2300a24c630eb5&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=3209a3d044b84a0ebd7cf1edf2da7120 => generated 259 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:09.159842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f85790-1561-417e-a1e7-018219f1d22c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:09.199524 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7524f-9184-4229-8d59-b443ba0b284a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:09.201057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7524f-9184-4229-8d59-b443ba0b284a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:09.201692 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7524f-9184-4229-8d59-b443ba0b284a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:09.202462 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7524f-9184-4229-8d59-b443ba0b284a None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:09.203470 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7524f-9184-4229-8d59-b443ba0b284a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:09.204046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7524f-9184-4229-8d59-b443ba0b284a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:09.205778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7524f-9184-4229-8d59-b443ba0b284a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:09.226164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 273/549] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 23 06:00:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:09.232705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087c9d3-0a74-4469-83a4-555d2a4e770c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:09.272836 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc28a3-f488-404a-a661-ef8c4b21fe46 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:09.273130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc28a3-f488-404a-a661-ef8c4b21fe46 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:09.273515 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc28a3-f488-404a-a661-ef8c4b21fe46 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:09.276542 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 274/550] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:00:09.282720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df333b95-2816-4eb9-8ae4-de236863a1d4 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:09.283000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df333b95-2816-4eb9-8ae4-de236863a1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:09.283457 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df333b95-2816-4eb9-8ae4-de236863a1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:09.322556 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-df333b95-2816-4eb9-8ae4-de236863a1d4 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:09.323764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-df333b95-2816-4eb9-8ae4-de236863a1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:09.398644 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087c9d3-0a74-4469-83a4-555d2a4e770c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:09.437412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:09.439087 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc4fab-0442-43af-9c8a-6a806e81512a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:09.439704 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc4fab-0442-43af-9c8a-6a806e81512a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:09.439979 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc4fab-0442-43af-9c8a-6a806e81512a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:09.440248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc4fab-0442-43af-9c8a-6a806e81512a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:09.467296 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:09.473215 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 275/551] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:09.514350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc4fab-0442-43af-9c8a-6a806e81512a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:09.515940 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc4fab-0442-43af-9c8a-6a806e81512a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:09.619569 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 277/552] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:00:09.625903 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df333b95-2816-4eb9-8ae4-de236863a1d4 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:09.756372 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df333b95-2816-4eb9-8ae4-de236863a1d4 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:09.793646 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37bf11-d422-40da-b3d4-8b20571a6c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:09.794891 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37bf11-d422-40da-b3d4-8b20571a6c6c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:09.795376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37bf11-d422-40da-b3d4-8b20571a6c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:09.795847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37bf11-d422-40da-b3d4-8b20571a6c6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:09.807439 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37bf11-d422-40da-b3d4-8b20571a6c6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:09.809229 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37bf11-d422-40da-b3d4-8b20571a6c6c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:09.822979 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82d971-079f-4829-819c-dd5f13146eaa None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:09.823270 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82d971-079f-4829-819c-dd5f13146eaa None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:09.823532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82d971-079f-4829-819c-dd5f13146eaa None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:09.825109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 278/553] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:09.830691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f2bec-387f-46b4-9145-40d2e54ced0a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:09.830966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f2bec-387f-46b4-9145-40d2e54ced0a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:09.831235 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f2bec-387f-46b4-9145-40d2e54ced0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-7a37bf11-d422-40da-b3d4-8b20571a6c6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:09.831837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:09.832860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:09.833723 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:09.833723 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:09.833723 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:09.835768 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 276/554] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:09.841949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37bf11-d422-40da-b3d4-8b20571a6c6c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:09.870823 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a95f2bec-387f-46b4-9145-40d2e54ced0a None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:09.871935 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a95f2bec-387f-46b4-9145-40d2e54ced0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:09.886722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:09.937628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:09.961897 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 279/555] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:09.962926 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f2bec-387f-46b4-9145-40d2e54ced0a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:09.972079 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37bf11-d422-40da-b3d4-8b20571a6c6c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:10.015020 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507d217-bb0e-487d-8fd3-578ae2099f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:10.016127 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507d217-bb0e-487d-8fd3-578ae2099f52 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:10.016649 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507d217-bb0e-487d-8fd3-578ae2099f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:10.017118 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507d217-bb0e-487d-8fd3-578ae2099f52 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:10.018005 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507d217-bb0e-487d-8fd3-578ae2099f52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:10.018507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507d217-bb0e-487d-8fd3-578ae2099f52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:10.020131 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507d217-bb0e-487d-8fd3-578ae2099f52 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:10.034202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 277/556] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:10.043031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507d217-bb0e-487d-8fd3-578ae2099f52 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:10.181194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f2bec-387f-46b4-9145-40d2e54ced0a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:10.182156 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:10.188364 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507d217-bb0e-487d-8fd3-578ae2099f52 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:10.229284 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44d7db-b6ac-497f-84e4-8dee8d245ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:10.231200 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44d7db-b6ac-497f-84e4-8dee8d245ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:10.232059 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44d7db-b6ac-497f-84e4-8dee8d245ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:10.232780 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44d7db-b6ac-497f-84e4-8dee8d245ff4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:10.242422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44d7db-b6ac-497f-84e4-8dee8d245ff4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:10.244558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44d7db-b6ac-497f-84e4-8dee8d245ff4 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:10.250500 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb1865-ff1c-447f-b958-190dcb2d8786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:10.251172 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb1865-ff1c-447f-b958-190dcb2d8786 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:10.251448 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb1865-ff1c-447f-b958-190dcb2d8786 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:10.251911 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb1865-ff1c-447f-b958-190dcb2d8786 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:10.260885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-ee44d7db-b6ac-497f-84e4-8dee8d245ff4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:10.260885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:10.260885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:10.260885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:10.260885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:10.260885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:10.260885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:10.260885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:10.260885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:10.260885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:10.260885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:10.260885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:10.260885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:10.260885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:10.260885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:10.262433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:10.264031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:10.264031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:10.264031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:10.264031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:10.264031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:10.264031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:10.264031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:10.264031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:10.266236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 278/557] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:10.272226 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac256a0-6509-41ce-aa14-f94da617038c None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:10.272387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb1865-ff1c-447f-b958-190dcb2d8786 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:10.272876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac256a0-6509-41ce-aa14-f94da617038c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:10.273328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac256a0-6509-41ce-aa14-f94da617038c None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-fddb1865-ff1c-447f-b958-190dcb2d8786 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:10.273545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:10.275129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:10.275948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:10.275948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:10.275948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:10.275948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:10.275948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:10.275948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:10.275948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:10.275948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:10.275948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:10.275948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:10.275948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:10.275948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:10.275948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 280/558] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:10.276646 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb1865-ff1c-447f-b958-190dcb2d8786 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:10.278650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 279/559] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:10 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:10.281577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac256a0-6509-41ce-aa14-f94da617038c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:10.346650 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:10.417111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb1865-ff1c-447f-b958-190dcb2d8786 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:10.426742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac256a0-6509-41ce-aa14-f94da617038c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:10.458076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4ceba-75ce-49b7-93e5-284bdaef33ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:10.459083 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4ceba-75ce-49b7-93e5-284bdaef33ff None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:10.459362 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4ceba-75ce-49b7-93e5-284bdaef33ff None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:10.459638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4ceba-75ce-49b7-93e5-284bdaef33ff None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:10.460351 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4ceba-75ce-49b7-93e5-284bdaef33ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:10.460674 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4ceba-75ce-49b7-93e5-284bdaef33ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:10.462187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4ceba-75ce-49b7-93e5-284bdaef33ff None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:10.469311 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f56243-0836-4455-a3e7-7f10bd9f5e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:10.470196 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f56243-0836-4455-a3e7-7f10bd9f5e71 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:10.470718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f56243-0836-4455-a3e7-7f10bd9f5e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:10.471166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f56243-0836-4455-a3e7-7f10bd9f5e71 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:10.472088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f56243-0836-4455-a3e7-7f10bd9f5e71 None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:10.472564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f56243-0836-4455-a3e7-7f10bd9f5e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:10.474237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f56243-0836-4455-a3e7-7f10bd9f5e71 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:10.477569 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 281/560] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:10.479688 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba382405-0b7b-4301-a43b-f5f814f603c2 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:10.480169 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba382405-0b7b-4301-a43b-f5f814f603c2 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:10.480649 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba382405-0b7b-4301-a43b-f5f814f603c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:10.489565 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 280/561] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:10.490889 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f56243-0836-4455-a3e7-7f10bd9f5e71 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:10.508302 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ba382405-0b7b-4301-a43b-f5f814f603c2 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:10.509837 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ba382405-0b7b-4301-a43b-f5f814f603c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:10.517443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:10.581603 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 282/562] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:10.586801 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba382405-0b7b-4301-a43b-f5f814f603c2 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:10.611585 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f56243-0836-4455-a3e7-7f10bd9f5e71 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:10.623998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:10.657242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abc3ce-3dc5-4268-aad9-69e575484bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:10.657934 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abc3ce-3dc5-4268-aad9-69e575484bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:10.658223 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abc3ce-3dc5-4268-aad9-69e575484bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:10.658497 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abc3ce-3dc5-4268-aad9-69e575484bd4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:10.673924 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45abc3ce-3dc5-4268-aad9-69e575484bd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-45abc3ce-3dc5-4268-aad9-69e575484bd4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:10.680574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:10.681507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:10.682363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:10.682363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:10.682363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:10.682363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:10.682363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:10.682363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:10.682363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:10.682363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:10.682363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:10.682363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:10.682363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:10.682363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:10.686668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 281/563] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:10.688221 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abc3ce-3dc5-4268-aad9-69e575484bd4 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:10.721285 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba382405-0b7b-4301-a43b-f5f814f603c2 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:10.769128 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bee392-cd86-45d9-b190-732e3a3bad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:10.770284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bee392-cd86-45d9-b190-732e3a3bad72 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:10.770772 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bee392-cd86-45d9-b190-732e3a3bad72 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:10.771485 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bee392-cd86-45d9-b190-732e3a3bad72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:10.772985 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bee392-cd86-45d9-b190-732e3a3bad72 None admin] RBAC: Adding query filter params (role.id=3209a3d044b84a0ebd7cf1edf2da7120, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=a73720a950944fa08bf1276ea78423b4) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:10.773580 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bee392-cd86-45d9-b190-732e3a3bad72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:10.775947 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bee392-cd86-45d9-b190-732e3a3bad72 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:10.784183 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 283/564] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:10 2022] GET /identity/v3/role_assignments?user.id=a73720a950944fa08bf1276ea78423b4&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=3209a3d044b84a0ebd7cf1edf2da7120 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:10.785859 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bee392-cd86-45d9-b190-732e3a3bad72 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:10.805876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:10.850966 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abc3ce-3dc5-4268-aad9-69e575484bd4 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:10.909164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7dd8b-06af-4681-8c95-6b0493468f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:10.909814 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7dd8b-06af-4681-8c95-6b0493468f9c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:10.910093 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7dd8b-06af-4681-8c95-6b0493468f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:10.910365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7dd8b-06af-4681-8c95-6b0493468f9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:10.925338 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7dd8b-06af-4681-8c95-6b0493468f9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:10.926914 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7dd8b-06af-4681-8c95-6b0493468f9c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:10.937648 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:10.942829 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bee392-cd86-45d9-b190-732e3a3bad72 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-42b7dd8b-06af-4681-8c95-6b0493468f9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:10.944277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:10.945188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:10.946060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:10.946060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:10.946060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:10.946060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 282/565] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:10.947168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7dd8b-06af-4681-8c95-6b0493468f9c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:10.992262 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7a88f-55ac-47b9-9cb4-3fefeb193e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:10.993205 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7a88f-55ac-47b9-9cb4-3fefeb193e82 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:10.993585 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7a88f-55ac-47b9-9cb4-3fefeb193e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:10.993957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7a88f-55ac-47b9-9cb4-3fefeb193e82 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:10.994803 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7a88f-55ac-47b9-9cb4-3fefeb193e82 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:10.995205 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7a88f-55ac-47b9-9cb4-3fefeb193e82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:10.996734 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7a88f-55ac-47b9-9cb4-3fefeb193e82 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:11.006567 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 284/566] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:11.007581 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7a88f-55ac-47b9-9cb4-3fefeb193e82 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:11.079976 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7dd8b-06af-4681-8c95-6b0493468f9c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:11.119180 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:11.127898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ccf19d-19c6-4871-9a3d-5829f16e55a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:11.128547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccf19d-19c6-4871-9a3d-5829f16e55a7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:11.128843 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccf19d-19c6-4871-9a3d-5829f16e55a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:11.129100 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccf19d-19c6-4871-9a3d-5829f16e55a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:11.130480 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7a88f-55ac-47b9-9cb4-3fefeb193e82 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:11.144179 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ccf19d-19c6-4871-9a3d-5829f16e55a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-65ccf19d-19c6-4871-9a3d-5829f16e55a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:11.152443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:11.153980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:11.154849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:11.154849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:11.154849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:11.154849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:11.154849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:11.154849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:11.154849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:11.154849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:11.154849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:11.154849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:11.154849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:11.154849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:11.158593 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 283/567] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:11.159676 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ccf19d-19c6-4871-9a3d-5829f16e55a7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:11.175970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764e89e-38eb-48a3-a566-725c74f18789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:11.176911 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764e89e-38eb-48a3-a566-725c74f18789 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:11.177387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764e89e-38eb-48a3-a566-725c74f18789 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:11.177846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764e89e-38eb-48a3-a566-725c74f18789 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:11.178745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764e89e-38eb-48a3-a566-725c74f18789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:11.179240 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764e89e-38eb-48a3-a566-725c74f18789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:11.180804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764e89e-38eb-48a3-a566-725c74f18789 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:11.194753 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 285/568] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:11.195922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764e89e-38eb-48a3-a566-725c74f18789 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:11.218451 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:11.296535 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ccf19d-19c6-4871-9a3d-5829f16e55a7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:11.311593 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764e89e-38eb-48a3-a566-725c74f18789 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:11.338706 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63189405-028f-48c1-b284-aa43335eaec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:11.339345 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63189405-028f-48c1-b284-aa43335eaec9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:11.339625 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63189405-028f-48c1-b284-aa43335eaec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:11.339898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63189405-028f-48c1-b284-aa43335eaec9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:11.348256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d823d2-9f21-4eae-95c5-ddb9b222d776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:11.349011 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d823d2-9f21-4eae-95c5-ddb9b222d776 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:11.349291 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d823d2-9f21-4eae-95c5-ddb9b222d776 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:11.349857 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d823d2-9f21-4eae-95c5-ddb9b222d776 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:11.350228 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d823d2-9f21-4eae-95c5-ddb9b222d776 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:11.350559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d823d2-9f21-4eae-95c5-ddb9b222d776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:11.351752 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63189405-028f-48c1-b284-aa43335eaec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:11.351926 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d823d2-9f21-4eae-95c5-ddb9b222d776 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:11.353197 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63189405-028f-48c1-b284-aa43335eaec9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:11.360470 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 286/569] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:11.361616 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d823d2-9f21-4eae-95c5-ddb9b222d776 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-63189405-028f-48c1-b284-aa43335eaec9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:11.374257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:11.375501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:11.376361 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:11.376361 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:11.376361 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:11.376361 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 284/570] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:11.376621 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc0b88-66dc-4c13-bb53-482906c0c277 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:11.377392 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc0b88-66dc-4c13-bb53-482906c0c277 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:11.377392 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc0b88-66dc-4c13-bb53-482906c0c277 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:11.378484 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 285/571] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:11.383039 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc0b88-66dc-4c13-bb53-482906c0c277 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:11.434471 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:11.466135 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:11.487562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d823d2-9f21-4eae-95c5-ddb9b222d776 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:11.496408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc0b88-66dc-4c13-bb53-482906c0c277 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:11.524864 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9650a1-d0c5-4bb7-ae7e-b8757e486db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:11.525749 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9650a1-d0c5-4bb7-ae7e-b8757e486db7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:11.526118 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9650a1-d0c5-4bb7-ae7e-b8757e486db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:11.526486 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9650a1-d0c5-4bb7-ae7e-b8757e486db7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:11.534076 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c33990-dd50-4c73-b83c-44293b090709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:11.534725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c33990-dd50-4c73-b83c-44293b090709 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:11.535004 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c33990-dd50-4c73-b83c-44293b090709 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:11.535218 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9650a1-d0c5-4bb7-ae7e-b8757e486db7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:11.535318 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c33990-dd50-4c73-b83c-44293b090709 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:11.536655 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9650a1-d0c5-4bb7-ae7e-b8757e486db7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:11.545985 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c33990-dd50-4c73-b83c-44293b090709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:11.547525 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c33990-dd50-4c73-b83c-44293b090709 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-1e9650a1-d0c5-4bb7-ae7e-b8757e486db7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:11.550212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:11.551334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:11.552337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:11.552337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:11.552337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:11.552337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:11.552337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:11.552337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:11.552337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:11.552337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:11.552337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:00:11.552337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:11.556089 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 287/572] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 23 06:00:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:11.556715 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9650a1-d0c5-4bb7-ae7e-b8757e486db7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-46c33990-dd50-4c73-b83c-44293b090709 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:11.562799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:11.563725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:11.569154 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:11.569154 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:11.569154 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:11.569154 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 286/573] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:11.569154 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301612ff-49d3-410a-921f-116d2849506b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:11.569154 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301612ff-49d3-410a-921f-116d2849506b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:11.569154 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301612ff-49d3-410a-921f-116d2849506b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:11.585244 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-301612ff-49d3-410a-921f-116d2849506b None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:11.586573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-301612ff-49d3-410a-921f-116d2849506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:11.659638 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 287/574] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:11.660489 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301612ff-49d3-410a-921f-116d2849506b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:11.678181 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9650a1-d0c5-4bb7-ae7e-b8757e486db7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:11.694273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:11.719939 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21661f-c84f-4a53-8c67-c06f0ce1c972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:11.720919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21661f-c84f-4a53-8c67-c06f0ce1c972 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:11.721380 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21661f-c84f-4a53-8c67-c06f0ce1c972 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:11.721830 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21661f-c84f-4a53-8c67-c06f0ce1c972 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:11.722679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21661f-c84f-4a53-8c67-c06f0ce1c972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:11.723158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21661f-c84f-4a53-8c67-c06f0ce1c972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:11.724810 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21661f-c84f-4a53-8c67-c06f0ce1c972 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:11.739655 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 288/575] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:11.740828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21661f-c84f-4a53-8c67-c06f0ce1c972 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:11.744012 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:11.775586 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301612ff-49d3-410a-921f-116d2849506b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:11.821071 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badeefa6-df0b-4271-9a10-aaf2cefa5a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:11.822825 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badeefa6-df0b-4271-9a10-aaf2cefa5a81 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:11.823293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badeefa6-df0b-4271-9a10-aaf2cefa5a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:11.824078 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badeefa6-df0b-4271-9a10-aaf2cefa5a81 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:11.825822 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badeefa6-df0b-4271-9a10-aaf2cefa5a81 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:11.826339 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badeefa6-df0b-4271-9a10-aaf2cefa5a81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:11.828267 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badeefa6-df0b-4271-9a10-aaf2cefa5a81 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:11.853569 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 288/576] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 23 06:00:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:11.854732 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badeefa6-df0b-4271-9a10-aaf2cefa5a81 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:11.876313 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21661f-c84f-4a53-8c67-c06f0ce1c972 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:11.920149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccf6f2-4ffd-4a7c-a1c7-93216911c75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:11.920999 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccf6f2-4ffd-4a7c-a1c7-93216911c75e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:11.921282 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccf6f2-4ffd-4a7c-a1c7-93216911c75e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:11.921557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccf6f2-4ffd-4a7c-a1c7-93216911c75e None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:11.922232 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccf6f2-4ffd-4a7c-a1c7-93216911c75e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:11.922633 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccf6f2-4ffd-4a7c-a1c7-93216911c75e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:11.924094 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccf6f2-4ffd-4a7c-a1c7-93216911c75e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:11.938183 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 289/577] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:11.939184 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccf6f2-4ffd-4a7c-a1c7-93216911c75e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:11.975255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:12.002264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badeefa6-df0b-4271-9a10-aaf2cefa5a81 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:12.011934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:12.039641 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2e648-0c69-4eb0-b2c1-4881519cb923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:12.040265 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2e648-0c69-4eb0-b2c1-4881519cb923 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:12.040539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2e648-0c69-4eb0-b2c1-4881519cb923 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:12.040811 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2e648-0c69-4eb0-b2c1-4881519cb923 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:12.049014 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2e648-0c69-4eb0-b2c1-4881519cb923 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:12.050447 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2e648-0c69-4eb0-b2c1-4881519cb923 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:12.056715 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccf6f2-4ffd-4a7c-a1c7-93216911c75e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-b4b2e648-0c69-4eb0-b2c1-4881519cb923 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:12.062607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:12.063543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:12.064520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:12.064520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:12.064520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:12.064520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:12.064520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:12.064520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:12.064520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:12.064520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:12.064520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 06:00:12.064520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:12.064520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 289/578] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:00:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:12.064520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2e648-0c69-4eb0-b2c1-4881519cb923 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:12.097610 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a9ffa-206e-435b-be8d-b04942353b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:12.098146 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a9ffa-206e-435b-be8d-b04942353b5e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:12.098437 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a9ffa-206e-435b-be8d-b04942353b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:12.098712 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a9ffa-206e-435b-be8d-b04942353b5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:12.109782 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a9ffa-206e-435b-be8d-b04942353b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:12.111277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a9ffa-206e-435b-be8d-b04942353b5e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-217a9ffa-206e-435b-be8d-b04942353b5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:12.125726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:12.126891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:12.127944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:12.127944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:12.127944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:12.131567 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 290/579] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:12.132205 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a9ffa-206e-435b-be8d-b04942353b5e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:12.173230 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2e648-0c69-4eb0-b2c1-4881519cb923 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:12.207946 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cf6ed-2236-4328-9b28-4f1503265c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:12.208572 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cf6ed-2236-4328-9b28-4f1503265c69 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:12.208845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cf6ed-2236-4328-9b28-4f1503265c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:12.209109 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cf6ed-2236-4328-9b28-4f1503265c69 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:12.217143 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5cf6ed-2236-4328-9b28-4f1503265c69 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:12.218594 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5cf6ed-2236-4328-9b28-4f1503265c69 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-af5cf6ed-2236-4328-9b28-4f1503265c69 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:12.229574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:12.230547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:12.236957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:12.236957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:12.236957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:12.236957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:12.236957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:12.236957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:12.236957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:12.236957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:12.236957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 06:00:12.236957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:12.236957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 290/580] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:00:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:12.236957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cf6ed-2236-4328-9b28-4f1503265c69 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:12.240164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:12.242348 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a9ffa-206e-435b-be8d-b04942353b5e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:12.257735 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:12.279707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a84d0c-dae6-4b97-9f43-5a0490656fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:12.280420 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a84d0c-dae6-4b97-9f43-5a0490656fde None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:12.280769 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a84d0c-dae6-4b97-9f43-5a0490656fde None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:12.281109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a84d0c-dae6-4b97-9f43-5a0490656fde None admin] PATH_INFO: `/v3/roles/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:12.292445 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a84d0c-dae6-4b97-9f43-5a0490656fde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-95a84d0c-dae6-4b97-9f43-5a0490656fde None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:12.298396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:12.299590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:12.300784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:12.300784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:12.300784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:12.300784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:12.300784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:12.300784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:12.300784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:12.300784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:12.300784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:12.300784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:12.300784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:12.300784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:12.301452 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 291/581] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:12.302149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a84d0c-dae6-4b97-9f43-5a0490656fde None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:12.364426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cf6ed-2236-4328-9b28-4f1503265c69 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:12.401883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71912c33-caa0-4ca0-b895-f98fd39b2e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:12.402495 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71912c33-caa0-4ca0-b895-f98fd39b2e8c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:12.402791 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71912c33-caa0-4ca0-b895-f98fd39b2e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:12.403057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71912c33-caa0-4ca0-b895-f98fd39b2e8c None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:12.403872 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71912c33-caa0-4ca0-b895-f98fd39b2e8c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:12.404177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71912c33-caa0-4ca0-b895-f98fd39b2e8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:12.405603 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71912c33-caa0-4ca0-b895-f98fd39b2e8c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:12.406970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a84d0c-dae6-4b97-9f43-5a0490656fde None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:12.426558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 291/582] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 23 06:00:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:12.426558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71912c33-caa0-4ca0-b895-f98fd39b2e8c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:12.450679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab79ed-816f-49c1-adf5-3c411dbb8336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:12.451280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab79ed-816f-49c1-adf5-3c411dbb8336 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:12.451550 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab79ed-816f-49c1-adf5-3c411dbb8336 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:12.451810 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab79ed-816f-49c1-adf5-3c411dbb8336 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:12.452446 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab79ed-816f-49c1-adf5-3c411dbb8336 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:12.452737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab79ed-816f-49c1-adf5-3c411dbb8336 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:12.454102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab79ed-816f-49c1-adf5-3c411dbb8336 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:12.470759 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 292/583] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:12.470759 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab79ed-816f-49c1-adf5-3c411dbb8336 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:12.502335 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:12.503536 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:12.548415 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71912c33-caa0-4ca0-b895-f98fd39b2e8c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:12.576059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab79ed-816f-49c1-adf5-3c411dbb8336 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:12.584535 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79984e9c-ce2d-4a0e-9142-4ae4cd931a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:12.585355 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79984e9c-ce2d-4a0e-9142-4ae4cd931a7c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:12.585786 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79984e9c-ce2d-4a0e-9142-4ae4cd931a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:12.586061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79984e9c-ce2d-4a0e-9142-4ae4cd931a7c None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:12.586820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79984e9c-ce2d-4a0e-9142-4ae4cd931a7c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:12.587128 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79984e9c-ce2d-4a0e-9142-4ae4cd931a7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:12.588549 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79984e9c-ce2d-4a0e-9142-4ae4cd931a7c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:12.608243 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 292/584] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 23 06:00:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:12.613012 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79984e9c-ce2d-4a0e-9142-4ae4cd931a7c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:12.615435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52683b76-de8f-4445-8bf5-3299135a49af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:12.616285 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52683b76-de8f-4445-8bf5-3299135a49af None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:12.617482 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52683b76-de8f-4445-8bf5-3299135a49af None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:12.618014 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52683b76-de8f-4445-8bf5-3299135a49af None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:12.618724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52683b76-de8f-4445-8bf5-3299135a49af None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:12.619031 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52683b76-de8f-4445-8bf5-3299135a49af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:12.620430 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52683b76-de8f-4445-8bf5-3299135a49af None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:12.631428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 293/585] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:12.636499 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52683b76-de8f-4445-8bf5-3299135a49af None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:12.722838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79984e9c-ce2d-4a0e-9142-4ae4cd931a7c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:12.743231 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:12.745265 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:12.752052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52683b76-de8f-4445-8bf5-3299135a49af None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:12.760772 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea15982-2c10-4933-9094-1f3cdd4f8c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:12.761531 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea15982-2c10-4933-9094-1f3cdd4f8c43 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:12.761807 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea15982-2c10-4933-9094-1f3cdd4f8c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:12.762074 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea15982-2c10-4933-9094-1f3cdd4f8c43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:12.772297 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea15982-2c10-4933-9094-1f3cdd4f8c43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:12.773745 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea15982-2c10-4933-9094-1f3cdd4f8c43 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-0ea15982-2c10-4933-9094-1f3cdd4f8c43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:12.788798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:12.789969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:12.791083 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:12.791083 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:12.791083 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:12.791083 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 293/586] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:12.791398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea15982-2c10-4933-9094-1f3cdd4f8c43 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:12.795559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1df40c-8bb1-450a-8fea-0cff21e465f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:12.796194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1df40c-8bb1-450a-8fea-0cff21e465f6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:12.796468 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1df40c-8bb1-450a-8fea-0cff21e465f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:12.796737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1df40c-8bb1-450a-8fea-0cff21e465f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:12.805301 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1df40c-8bb1-450a-8fea-0cff21e465f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:12.806787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1df40c-8bb1-450a-8fea-0cff21e465f6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:12.818080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-7a1df40c-8bb1-450a-8fea-0cff21e465f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:12.818080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:12.818080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:12.818080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:12.818080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:12.818080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:12.818080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:12.818080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:12.818080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:12.818080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:12.818080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:12.818080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:12.818080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:12.818080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:12.818080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:12.819176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:12.820045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:12.820045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:12.820045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:12.820045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:12.820045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:12.820045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:12.820045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:12.820045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:12.820045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 294/587] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:12.820045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1df40c-8bb1-450a-8fea-0cff21e465f6 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:12.898899 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea15982-2c10-4933-9094-1f3cdd4f8c43 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:12.921621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1df40c-8bb1-450a-8fea-0cff21e465f6 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:12.936448 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40100e-95fe-49dc-8b42-b7f316d7d741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:12.937084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40100e-95fe-49dc-8b42-b7f316d7d741 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:12.937359 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40100e-95fe-49dc-8b42-b7f316d7d741 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:12.937631 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40100e-95fe-49dc-8b42-b7f316d7d741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:12.949642 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40100e-95fe-49dc-8b42-b7f316d7d741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:12.950273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40100e-95fe-49dc-8b42-b7f316d7d741 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:12.961677 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e51de1-93a7-4d24-8b36-ff3e9229fbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:12.962302 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e51de1-93a7-4d24-8b36-ff3e9229fbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:12.962615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e51de1-93a7-4d24-8b36-ff3e9229fbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:12.962892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e51de1-93a7-4d24-8b36-ff3e9229fbf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-2f40100e-95fe-49dc-8b42-b7f316d7d741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:12.964958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:12.966539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:12.967689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:12.967689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:12.967689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:12.967689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 294/588] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:12.968233 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d230f-e4ce-4e49-b0e2-8f86e602c8cb None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:12.968515 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d230f-e4ce-4e49-b0e2-8f86e602c8cb None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:12.968785 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d230f-e4ce-4e49-b0e2-8f86e602c8cb None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:12.970018 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 295/589] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:12.970744 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d230f-e4ce-4e49-b0e2-8f86e602c8cb None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:12.978881 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e51de1-93a7-4d24-8b36-ff3e9229fbf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:12.980325 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e51de1-93a7-4d24-8b36-ff3e9229fbf9 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-b1e51de1-93a7-4d24-8b36-ff3e9229fbf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:12.995109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:12.996187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:12.997276 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:12.997276 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:12.997276 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:12.997276 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 295/590] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:13.000235 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e51de1-93a7-4d24-8b36-ff3e9229fbf9 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:13.005055 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:13.011588 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:13.084665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d230f-e4ce-4e49-b0e2-8f86e602c8cb None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:13.100343 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e51de1-93a7-4d24-8b36-ff3e9229fbf9 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:13.119034 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251dc24-d1c1-4c87-8805-a8eb68e29379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:13.119632 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251dc24-d1c1-4c87-8805-a8eb68e29379 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:13.119904 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251dc24-d1c1-4c87-8805-a8eb68e29379 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:13.120169 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251dc24-d1c1-4c87-8805-a8eb68e29379 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:13.120796 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251dc24-d1c1-4c87-8805-a8eb68e29379 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:13.121109 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251dc24-d1c1-4c87-8805-a8eb68e29379 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:13.122634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251dc24-d1c1-4c87-8805-a8eb68e29379 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:13.134372 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441bc855-1866-4749-b9c6-d5e88670a2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:13.134549 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 296/591] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:13.135001 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441bc855-1866-4749-b9c6-d5e88670a2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:13.135279 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441bc855-1866-4749-b9c6-d5e88670a2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:13.135600 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251dc24-d1c1-4c87-8805-a8eb68e29379 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:13.135722 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441bc855-1866-4749-b9c6-d5e88670a2c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:13.136313 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441bc855-1866-4749-b9c6-d5e88670a2c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:13.136622 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441bc855-1866-4749-b9c6-d5e88670a2c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:13.138104 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441bc855-1866-4749-b9c6-d5e88670a2c9 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:13.150831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 296/592] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:13.152022 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04d7a6-342b-481d-8665-3fc6f564acef None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:13.152307 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04d7a6-342b-481d-8665-3fc6f564acef None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:13.152576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04d7a6-342b-481d-8665-3fc6f564acef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:13.171044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ab04d7a6-342b-481d-8665-3fc6f564acef None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:13.172252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ab04d7a6-342b-481d-8665-3fc6f564acef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:13.225404 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:13.237032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:13.242363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251dc24-d1c1-4c87-8805-a8eb68e29379 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:13.244341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 297/593] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:13.245155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04d7a6-342b-481d-8665-3fc6f564acef None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:13.279180 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beda7cb-5095-4bd6-a2fa-2c37c1bf929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:13.279793 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beda7cb-5095-4bd6-a2fa-2c37c1bf929d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:13.280064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beda7cb-5095-4bd6-a2fa-2c37c1bf929d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:13.280328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beda7cb-5095-4bd6-a2fa-2c37c1bf929d None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:13.281015 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beda7cb-5095-4bd6-a2fa-2c37c1bf929d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:13.281316 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beda7cb-5095-4bd6-a2fa-2c37c1bf929d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:13.282699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beda7cb-5095-4bd6-a2fa-2c37c1bf929d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:13.293977 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 297/594] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:13.294986 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beda7cb-5095-4bd6-a2fa-2c37c1bf929d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:13.364049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04d7a6-342b-481d-8665-3fc6f564acef None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:13.405480 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beda7cb-5095-4bd6-a2fa-2c37c1bf929d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:13.406021 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6a78a-b5a4-4b1e-a5ba-e4a77ed79f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:13.406709 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6a78a-b5a4-4b1e-a5ba-e4a77ed79f42 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:13.407053 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6a78a-b5a4-4b1e-a5ba-e4a77ed79f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:13.407392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6a78a-b5a4-4b1e-a5ba-e4a77ed79f42 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:13.408833 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6a78a-b5a4-4b1e-a5ba-e4a77ed79f42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:13.408833 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6a78a-b5a4-4b1e-a5ba-e4a77ed79f42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:13.409738 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6a78a-b5a4-4b1e-a5ba-e4a77ed79f42 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:13.425794 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 298/595] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:13.425794 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6a78a-b5a4-4b1e-a5ba-e4a77ed79f42 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:13.446423 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5c4f7-bc29-4408-94e8-f66f86eb8aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:13.447171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5c4f7-bc29-4408-94e8-f66f86eb8aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:13.447526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5c4f7-bc29-4408-94e8-f66f86eb8aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:13.447937 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5c4f7-bc29-4408-94e8-f66f86eb8aa3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:13.456018 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5c4f7-bc29-4408-94e8-f66f86eb8aa3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:13.457454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5c4f7-bc29-4408-94e8-f66f86eb8aa3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:13.470961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-83d5c4f7-bc29-4408-94e8-f66f86eb8aa3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:13.470961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:13.470961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:13.470961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:13.470961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:13.470961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:13.470961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:13.470961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:13.470961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:13.470961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:13.470961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:13.470961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:13.470961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:13.470961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:13.470961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:13.472117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:13.473260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:13.473260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:13.473260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:13.473260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:13.473260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:13.473260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:13.473260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:13.473260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:13.474646 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 298/596] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:13.476358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5c4f7-bc29-4408-94e8-f66f86eb8aa3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:13.481998 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:13.489734 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:13.536047 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6a78a-b5a4-4b1e-a5ba-e4a77ed79f42 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:13.571875 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bb36a-b8ef-40a2-8086-c0014b328d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:13.572525 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bb36a-b8ef-40a2-8086-c0014b328d6c None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:13.572804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bb36a-b8ef-40a2-8086-c0014b328d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:13.573074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bb36a-b8ef-40a2-8086-c0014b328d6c None admin] PATH_INFO: `/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/91267a049343452eb208205ee3ea27c4/roles/3209a3d044b84a0ebd7cf1edf2da7120` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:13.583877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5c4f7-bc29-4408-94e8-f66f86eb8aa3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:13.588234 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bb36a-b8ef-40a2-8086-c0014b328d6c None admin] RBAC: Authorizing `identity:create_grant(project_id=b0b43dead1664475b9dd52113e08d7ed, user_id=91267a049343452eb208205ee3ea27c4, role_id=3209a3d044b84a0ebd7cf1edf2da7120)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:13.589674 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bb36a-b8ef-40a2-8086-c0014b328d6c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:13.614146 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 299/597] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:00:13 2022] PUT /identity/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/91267a049343452eb208205ee3ea27c4/roles/3209a3d044b84a0ebd7cf1edf2da7120 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:00:13.614146 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bb36a-b8ef-40a2-8086-c0014b328d6c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:13.620472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a609d8-173d-4ae5-8b88-6e49e0ffa255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:13.621144 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a609d8-173d-4ae5-8b88-6e49e0ffa255 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:13.621426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a609d8-173d-4ae5-8b88-6e49e0ffa255 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:13.621688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a609d8-173d-4ae5-8b88-6e49e0ffa255 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:13.631387 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a609d8-173d-4ae5-8b88-6e49e0ffa255 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:13.633778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a609d8-173d-4ae5-8b88-6e49e0ffa255 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:13.650072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-e2a609d8-173d-4ae5-8b88-6e49e0ffa255 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:13.650072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:13.650072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:13.650072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:13.650072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:13.650072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:13.650072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:13.650072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:13.650072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:13.650072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:13.650072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:13.650072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:13.650072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:13.650072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:13.650072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:13.651282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:13.657353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:13.657353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:13.657353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:13.657353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:13.657353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:13.657353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:13.657353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:13.657353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:13.657353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 299/598] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:13.657353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a609d8-173d-4ae5-8b88-6e49e0ffa255 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:13.737098 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:13.749526 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bb36a-b8ef-40a2-8086-c0014b328d6c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:13.763569 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:13.790605 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a609d8-173d-4ae5-8b88-6e49e0ffa255 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:13.792309 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97440e-eb36-45ec-9970-02306df774c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:13.793008 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97440e-eb36-45ec-9970-02306df774c4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:13.793288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97440e-eb36-45ec-9970-02306df774c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:13.793557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97440e-eb36-45ec-9970-02306df774c4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:13.801705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97440e-eb36-45ec-9970-02306df774c4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:13.803186 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97440e-eb36-45ec-9970-02306df774c4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-7d97440e-eb36-45ec-9970-02306df774c4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:13.815145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:13.821182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:13.822365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:13.822365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:13.822365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:13.822365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:13.822365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:13.822365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:13.822365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:13.822365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:13.822365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 06:00:13.822365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:13.822365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 300/599] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:13.822365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97440e-eb36-45ec-9970-02306df774c4 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:13.830729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a55585-28e1-48c1-8959-e21b7f40e41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:13.831424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a55585-28e1-48c1-8959-e21b7f40e41a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:13.831765 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a55585-28e1-48c1-8959-e21b7f40e41a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:13.832413 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a55585-28e1-48c1-8959-e21b7f40e41a None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:13.833194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a55585-28e1-48c1-8959-e21b7f40e41a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:13.833870 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a55585-28e1-48c1-8959-e21b7f40e41a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:13.835672 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a55585-28e1-48c1-8959-e21b7f40e41a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:13.853943 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 300/600] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:13.853943 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a55585-28e1-48c1-8959-e21b7f40e41a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:13.935394 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97440e-eb36-45ec-9970-02306df774c4 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:13.972202 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310eed0c-6abc-46f1-91e4-ace161476135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:13.972912 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eed0c-6abc-46f1-91e4-ace161476135 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:13.973270 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eed0c-6abc-46f1-91e4-ace161476135 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:13.973535 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eed0c-6abc-46f1-91e4-ace161476135 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:13.978236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a55585-28e1-48c1-8959-e21b7f40e41a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:13.986119 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310eed0c-6abc-46f1-91e4-ace161476135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-310eed0c-6abc-46f1-91e4-ace161476135 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:13.993110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:13.994899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:13.997880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:13.997880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:13.997880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:13.997880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:13.997880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:13.997880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:13.997880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:13.997880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:13.997880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:13.997880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:13.997880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:13.997880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:13.997880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 301/601] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:13.997880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310eed0c-6abc-46f1-91e4-ace161476135 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:14.026127 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:14.040925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5133ca-4206-49af-a50a-48df79ec78c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:14.041801 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5133ca-4206-49af-a50a-48df79ec78c6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:14.042007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5133ca-4206-49af-a50a-48df79ec78c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:14.042290 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5133ca-4206-49af-a50a-48df79ec78c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:14.043030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5133ca-4206-49af-a50a-48df79ec78c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:14.043339 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5133ca-4206-49af-a50a-48df79ec78c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:14.043523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:14.045077 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5133ca-4206-49af-a50a-48df79ec78c6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:14.058729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 301/602] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:14.063388 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5133ca-4206-49af-a50a-48df79ec78c6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:14.128083 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310eed0c-6abc-46f1-91e4-ace161476135 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:14.163604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba44f9-8f53-4df1-81b3-99b4a1ce20d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:14.164207 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba44f9-8f53-4df1-81b3-99b4a1ce20d2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:14.164489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba44f9-8f53-4df1-81b3-99b4a1ce20d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:14.164757 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba44f9-8f53-4df1-81b3-99b4a1ce20d2 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:14.165483 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba44f9-8f53-4df1-81b3-99b4a1ce20d2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:14.165790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba44f9-8f53-4df1-81b3-99b4a1ce20d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:14.167260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba44f9-8f53-4df1-81b3-99b4a1ce20d2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:14.167898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5133ca-4206-49af-a50a-48df79ec78c6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:14.185254 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 302/603] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Aug 23 06:00:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:14.185869 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba44f9-8f53-4df1-81b3-99b4a1ce20d2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:14.207629 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d115c4-5e15-4ea8-98ab-57f8ee3d0388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:14.208427 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d115c4-5e15-4ea8-98ab-57f8ee3d0388 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:14.208802 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d115c4-5e15-4ea8-98ab-57f8ee3d0388 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:14.209145 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d115c4-5e15-4ea8-98ab-57f8ee3d0388 None admin] PATH_INFO: `/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/89621d839d1c44798f691db4d49e1278/roles/3209a3d044b84a0ebd7cf1edf2da7120` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:14.225223 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d115c4-5e15-4ea8-98ab-57f8ee3d0388 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0b43dead1664475b9dd52113e08d7ed, user_id=89621d839d1c44798f691db4d49e1278, role_id=3209a3d044b84a0ebd7cf1edf2da7120)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:14.226701 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d115c4-5e15-4ea8-98ab-57f8ee3d0388 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:14.242256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 302/604] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:00:14 2022] PUT /identity/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/89621d839d1c44798f691db4d49e1278/roles/3209a3d044b84a0ebd7cf1edf2da7120 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:00:14.243378 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d115c4-5e15-4ea8-98ab-57f8ee3d0388 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:14.289425 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:14.301266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:14.308761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba44f9-8f53-4df1-81b3-99b4a1ce20d2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:14.351878 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d4d97-1315-4959-8c7e-3abccbe0b485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:14.352492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d4d97-1315-4959-8c7e-3abccbe0b485 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:14.352768 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d4d97-1315-4959-8c7e-3abccbe0b485 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:14.353036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d4d97-1315-4959-8c7e-3abccbe0b485 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:14.353897 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d4d97-1315-4959-8c7e-3abccbe0b485 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:14.354199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d4d97-1315-4959-8c7e-3abccbe0b485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:14.357468 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d4d97-1315-4959-8c7e-3abccbe0b485 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:14.371589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 303/605] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:14.373149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d4d97-1315-4959-8c7e-3abccbe0b485 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:14.417925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d115c4-5e15-4ea8-98ab-57f8ee3d0388 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:14.466639 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e88c1-df50-4408-a081-b240cbf8ed91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:14.466639 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e88c1-df50-4408-a081-b240cbf8ed91 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:14.466639 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e88c1-df50-4408-a081-b240cbf8ed91 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:14.466639 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e88c1-df50-4408-a081-b240cbf8ed91 None admin] PATH_INFO: `/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/c741b92684794c338d63913774ab841a/roles/3209a3d044b84a0ebd7cf1edf2da7120` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:14.480149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e88c1-df50-4408-a081-b240cbf8ed91 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0b43dead1664475b9dd52113e08d7ed, user_id=c741b92684794c338d63913774ab841a, role_id=3209a3d044b84a0ebd7cf1edf2da7120)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:14.481914 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e88c1-df50-4408-a081-b240cbf8ed91 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:14.500071 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 303/606] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:00:14 2022] PUT /identity/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/c741b92684794c338d63913774ab841a/roles/3209a3d044b84a0ebd7cf1edf2da7120 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:00:14.500988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e88c1-df50-4408-a081-b240cbf8ed91 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:14.505142 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d4d97-1315-4959-8c7e-3abccbe0b485 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:14.548572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf599ee8-9502-423a-a402-ac3e941e1b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:14.549228 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf599ee8-9502-423a-a402-ac3e941e1b28 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:14.549501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf599ee8-9502-423a-a402-ac3e941e1b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:14.549770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf599ee8-9502-423a-a402-ac3e941e1b28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:14.563788 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf599ee8-9502-423a-a402-ac3e941e1b28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:14.565251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf599ee8-9502-423a-a402-ac3e941e1b28 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:14.574962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-bf599ee8-9502-423a-a402-ac3e941e1b28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:14.584337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:14.585532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:14.586641 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:14.586641 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:14.586641 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:14.591365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 304/607] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:14.592758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf599ee8-9502-423a-a402-ac3e941e1b28 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:14.714232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:14.737562 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e88c1-df50-4408-a081-b240cbf8ed91 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:14.749498 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf599ee8-9502-423a-a402-ac3e941e1b28 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:14.787554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5359a-334b-47a3-840d-a1838363cf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:14.788469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5359a-334b-47a3-840d-a1838363cf19 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:14.788750 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5359a-334b-47a3-840d-a1838363cf19 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:14.789468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5359a-334b-47a3-840d-a1838363cf19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:14.798484 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7180c9-3d52-4820-9e30-383a4ba36f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:14.799121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7180c9-3d52-4820-9e30-383a4ba36f4d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:14.799397 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7180c9-3d52-4820-9e30-383a4ba36f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:14.799685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7180c9-3d52-4820-9e30-383a4ba36f4d None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:14.800390 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7180c9-3d52-4820-9e30-383a4ba36f4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:14.800732 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7180c9-3d52-4820-9e30-383a4ba36f4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:14.802187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7180c9-3d52-4820-9e30-383a4ba36f4d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:14.802720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5359a-334b-47a3-840d-a1838363cf19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:14.804212 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5359a-334b-47a3-840d-a1838363cf19 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:14.817207 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 305/608] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-81c5359a-334b-47a3-840d-a1838363cf19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:14.822354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:14.823573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:14.824657 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:14.824657 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:14.824657 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:14.824831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7180c9-3d52-4820-9e30-383a4ba36f4d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:14.824902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 304/609] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:14.830045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5359a-334b-47a3-840d-a1838363cf19 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:14.887079 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:14.931878 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7180c9-3d52-4820-9e30-383a4ba36f4d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:14.957995 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5359a-334b-47a3-840d-a1838363cf19 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:14.973445 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a01c96-ee4a-4838-a302-8f1e0b826806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:14.974324 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a01c96-ee4a-4838-a302-8f1e0b826806 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:14.974762 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a01c96-ee4a-4838-a302-8f1e0b826806 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:14.975166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a01c96-ee4a-4838-a302-8f1e0b826806 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:14.982845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a01c96-ee4a-4838-a302-8f1e0b826806 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:14.984481 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a01c96-ee4a-4838-a302-8f1e0b826806 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:14.996263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-63a01c96-ee4a-4838-a302-8f1e0b826806 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:14.996263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:14.996263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:14.996263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:14.996263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:14.996263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:14.996263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:14.996263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:14.996263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:14.996263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:14.996263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:14.996263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:14.996263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:14.996263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:14.996263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:14.997379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:14.998252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:14.998252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:14.998252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:14.998252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:14.998252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:14.998252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:14.998252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:14.998252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:15.004520 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 306/610] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:15.005975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a01c96-ee4a-4838-a302-8f1e0b826806 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:15.025662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4542ff-2b89-437e-91b8-a4d8cedf7f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:15.026281 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4542ff-2b89-437e-91b8-a4d8cedf7f79 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:15.026714 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4542ff-2b89-437e-91b8-a4d8cedf7f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:15.027008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4542ff-2b89-437e-91b8-a4d8cedf7f79 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:15.027762 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4542ff-2b89-437e-91b8-a4d8cedf7f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:15.028166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4542ff-2b89-437e-91b8-a4d8cedf7f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:15.029668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4542ff-2b89-437e-91b8-a4d8cedf7f79 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:15.042877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:15.047970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 305/611] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:15.048944 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4542ff-2b89-437e-91b8-a4d8cedf7f79 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:15.115368 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a01c96-ee4a-4838-a302-8f1e0b826806 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:15.139242 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:15.155365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6bf81-337a-4a46-8bc0-2240eab23689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:15.156226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6bf81-337a-4a46-8bc0-2240eab23689 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:15.156643 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6bf81-337a-4a46-8bc0-2240eab23689 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:15.157051 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6bf81-337a-4a46-8bc0-2240eab23689 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:15.157986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6bf81-337a-4a46-8bc0-2240eab23689 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:15.158467 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6bf81-337a-4a46-8bc0-2240eab23689 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:15.160160 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6bf81-337a-4a46-8bc0-2240eab23689 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:15.177575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 307/612] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:15.179094 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6bf81-337a-4a46-8bc0-2240eab23689 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:15.276270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4542ff-2b89-437e-91b8-a4d8cedf7f79 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:15.333442 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6bf81-337a-4a46-8bc0-2240eab23689 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:15.359255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e420a3-1e73-4c32-8c97-fce815606089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:15.359902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e420a3-1e73-4c32-8c97-fce815606089 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:15.360182 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e420a3-1e73-4c32-8c97-fce815606089 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:15.360447 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e420a3-1e73-4c32-8c97-fce815606089 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:15.371804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191dfda6-676b-421a-9b67-007acd53fe05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:15.373252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191dfda6-676b-421a-9b67-007acd53fe05 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:15.373669 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191dfda6-676b-421a-9b67-007acd53fe05 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:15.374136 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191dfda6-676b-421a-9b67-007acd53fe05 None admin] PATH_INFO: `/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/dae9eff3bbf04496af2300a24c630eb5/roles/3209a3d044b84a0ebd7cf1edf2da7120` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:15.379626 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e420a3-1e73-4c32-8c97-fce815606089 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:15.381076 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e420a3-1e73-4c32-8c97-fce815606089 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:15.390809 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191dfda6-676b-421a-9b67-007acd53fe05 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0b43dead1664475b9dd52113e08d7ed, user_id=dae9eff3bbf04496af2300a24c630eb5, role_id=3209a3d044b84a0ebd7cf1edf2da7120)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:15.393177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191dfda6-676b-421a-9b67-007acd53fe05 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-74e420a3-1e73-4c32-8c97-fce815606089 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:15.413840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:15.415158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:15.416159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:15.416159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:15.416159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:15.416159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:15.416159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:15.416159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:15.416159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:15.416159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:15.416159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:00:15.416159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:15.416159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 306/614] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:00:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:15.416820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 308/614] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:00:15 2022] PUT /identity/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/dae9eff3bbf04496af2300a24c630eb5/roles/3209a3d044b84a0ebd7cf1edf2da7120 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 06:00:15.423689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e420a3-1e73-4c32-8c97-fce815606089 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:15.507581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:15.562051 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e420a3-1e73-4c32-8c97-fce815606089 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:15.597115 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f2c7e-f00d-4000-ac19-f2b401fa866f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:15.597818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f2c7e-f00d-4000-ac19-f2b401fa866f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:15.598161 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f2c7e-f00d-4000-ac19-f2b401fa866f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:15.598491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f2c7e-f00d-4000-ac19-f2b401fa866f None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:15.599310 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f2c7e-f00d-4000-ac19-f2b401fa866f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:15.599681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f2c7e-f00d-4000-ac19-f2b401fa866f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:15.601174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f2c7e-f00d-4000-ac19-f2b401fa866f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:15.621194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 307/615] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 23 06:00:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:15.627057 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191dfda6-676b-421a-9b67-007acd53fe05 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:15.684662 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:15.718565 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95f798-8514-46d4-b404-74b5e04d2e9d None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:15.718992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95f798-8514-46d4-b404-74b5e04d2e9d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:15.719382 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95f798-8514-46d4-b404-74b5e04d2e9d None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:15.721193 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 308/616] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:15.726931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cda9d0-4353-4b54-a69d-5f483cd1140a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:15.727317 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cda9d0-4353-4b54-a69d-5f483cd1140a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:15.727685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cda9d0-4353-4b54-a69d-5f483cd1140a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:15.760454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-39cda9d0-4353-4b54-a69d-5f483cd1140a None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:15.761921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-39cda9d0-4353-4b54-a69d-5f483cd1140a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:15.774811 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191dfda6-676b-421a-9b67-007acd53fe05 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:15.819385 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7e086-94ce-4c80-95ba-43496e995f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:15.820328 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7e086-94ce-4c80-95ba-43496e995f09 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:15.820765 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7e086-94ce-4c80-95ba-43496e995f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:15.821216 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7e086-94ce-4c80-95ba-43496e995f09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:15.834760 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f7e086-94ce-4c80-95ba-43496e995f09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:15.836452 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f7e086-94ce-4c80-95ba-43496e995f09 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-75f7e086-94ce-4c80-95ba-43496e995f09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:15.855352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:15.857318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:15.858763 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:15.858763 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:15.858763 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:15.860885 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 309/617] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:15.866494 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7e086-94ce-4c80-95ba-43496e995f09 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:15.908334 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 309/618] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 23 06:00:15.964733 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:15.985308 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7e086-94ce-4c80-95ba-43496e995f09 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:16.024795 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14e4ad-045c-40a2-96e9-01be362afb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:16.025455 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14e4ad-045c-40a2-96e9-01be362afb15 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:16.025741 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14e4ad-045c-40a2-96e9-01be362afb15 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:16.026011 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14e4ad-045c-40a2-96e9-01be362afb15 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:16.026747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14e4ad-045c-40a2-96e9-01be362afb15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:16.027054 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14e4ad-045c-40a2-96e9-01be362afb15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:16.028554 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14e4ad-045c-40a2-96e9-01be362afb15 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:16.041312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 310/619] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:16.046789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cda9d0-4353-4b54-a69d-5f483cd1140a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:16.128937 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:16.195089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cda9d0-4353-4b54-a69d-5f483cd1140a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:16.239582 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71219f34-0a32-463c-835e-48dc9c1b5b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:16.240532 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71219f34-0a32-463c-835e-48dc9c1b5b07 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:16.240974 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71219f34-0a32-463c-835e-48dc9c1b5b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:16.241467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71219f34-0a32-463c-835e-48dc9c1b5b07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:16.255623 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71219f34-0a32-463c-835e-48dc9c1b5b07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:16.257298 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71219f34-0a32-463c-835e-48dc9c1b5b07 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:16.271424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-71219f34-0a32-463c-835e-48dc9c1b5b07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:16.271424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:16.271424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:16.271424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:16.271424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:16.271424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:16.271424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:16.271424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:16.271424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:16.271424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:16.271424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:16.271424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:16.271424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:16.271424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:16.271424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:16.272633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:16.273685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:16.273685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:16.273685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:16.273685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:16.273685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:16.273685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:16.273685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:16.273685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:16.275408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 310/620] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:16.281766 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14e4ad-045c-40a2-96e9-01be362afb15 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:16.328913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71219f34-0a32-463c-835e-48dc9c1b5b07 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:16.440841 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71219f34-0a32-463c-835e-48dc9c1b5b07 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:16.443376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:16.478704 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a5454-1716-4fbc-a5ea-031cbd8d4c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:16.479342 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a5454-1716-4fbc-a5ea-031cbd8d4c5a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:16.479624 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a5454-1716-4fbc-a5ea-031cbd8d4c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:16.479896 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a5454-1716-4fbc-a5ea-031cbd8d4c5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:16.491811 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a5454-1716-4fbc-a5ea-031cbd8d4c5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-665a5454-1716-4fbc-a5ea-031cbd8d4c5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:16.497231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:16.498185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:16.500328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:16.500328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:16.500328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:16.500328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:16.500328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:16.500328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:16.500328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:16.500328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:16.500328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:16.500328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:16.500328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:16.500328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:16.500328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 311/621] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:16.504570 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a5454-1716-4fbc-a5ea-031cbd8d4c5a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:16.551243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14e4ad-045c-40a2-96e9-01be362afb15 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:16.611886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a5454-1716-4fbc-a5ea-031cbd8d4c5a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:16.637052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5fdac-0ea0-4bea-99bb-382b657b9aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:16.643516 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5fdac-0ea0-4bea-99bb-382b657b9aec None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:16.644052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5fdac-0ea0-4bea-99bb-382b657b9aec None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:16.644538 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5fdac-0ea0-4bea-99bb-382b657b9aec None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:16.645461 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a5fdac-0ea0-4bea-99bb-382b657b9aec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:16.645993 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a5fdac-0ea0-4bea-99bb-382b657b9aec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:16.647697 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a5fdac-0ea0-4bea-99bb-382b657b9aec None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:16.652992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb72ef-04b4-457b-970e-7b2f9c7344c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:16.652992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb72ef-04b4-457b-970e-7b2f9c7344c7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:16.652992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb72ef-04b4-457b-970e-7b2f9c7344c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:16.652992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb72ef-04b4-457b-970e-7b2f9c7344c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:16.653913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb72ef-04b4-457b-970e-7b2f9c7344c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:16.653913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb72ef-04b4-457b-970e-7b2f9c7344c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:16.654924 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb72ef-04b4-457b-970e-7b2f9c7344c7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:16.663941 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 312/622] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:16.674397 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb72ef-04b4-457b-970e-7b2f9c7344c7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:16.683419 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 311/623] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:16.689087 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5fdac-0ea0-4bea-99bb-382b657b9aec None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:16.696414 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:16.700409 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:16.783368 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb72ef-04b4-457b-970e-7b2f9c7344c7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:16.818225 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5fdac-0ea0-4bea-99bb-382b657b9aec None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:16.819654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476fc2d-e038-4b7d-b152-382eea483cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:16.820298 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476fc2d-e038-4b7d-b152-382eea483cef None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:16.820577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476fc2d-e038-4b7d-b152-382eea483cef None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:16.820848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476fc2d-e038-4b7d-b152-382eea483cef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:16.832489 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e476fc2d-e038-4b7d-b152-382eea483cef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:16.833944 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e476fc2d-e038-4b7d-b152-382eea483cef None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-e476fc2d-e038-4b7d-b152-382eea483cef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:16.849885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:16.850952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:16.853154 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:16.853154 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:16.853154 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:16.854627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 313/624] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:16.855901 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a96edf-43eb-488b-a504-29aa117c5139 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:16.856239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a96edf-43eb-488b-a504-29aa117c5139 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:16.856586 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a96edf-43eb-488b-a504-29aa117c5139 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:16.859781 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 314/625] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:16.862024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a96edf-43eb-488b-a504-29aa117c5139 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:16.862861 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efd28c-069a-48ef-9030-ee93a85a623c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:16.863713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efd28c-069a-48ef-9030-ee93a85a623c None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:16.864146 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efd28c-069a-48ef-9030-ee93a85a623c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:16.864570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efd28c-069a-48ef-9030-ee93a85a623c None admin] PATH_INFO: `/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/a73720a950944fa08bf1276ea78423b4/roles/3209a3d044b84a0ebd7cf1edf2da7120` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:16.882431 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efd28c-069a-48ef-9030-ee93a85a623c None admin] RBAC: Authorizing `identity:create_grant(project_id=b0b43dead1664475b9dd52113e08d7ed, user_id=a73720a950944fa08bf1276ea78423b4, role_id=3209a3d044b84a0ebd7cf1edf2da7120)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:16.884264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efd28c-069a-48ef-9030-ee93a85a623c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:16.901526 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 312/626] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:00:16 2022] PUT /identity/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/a73720a950944fa08bf1276ea78423b4/roles/3209a3d044b84a0ebd7cf1edf2da7120 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 06:00:16.904257 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3024c82-0cea-42c5-a073-d8bd417791ce None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:16.904752 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3024c82-0cea-42c5-a073-d8bd417791ce None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:16.905195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3024c82-0cea-42c5-a073-d8bd417791ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:16.929390 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b3024c82-0cea-42c5-a073-d8bd417791ce None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:16.930881 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b3024c82-0cea-42c5-a073-d8bd417791ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:16.971267 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:17.003742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a96edf-43eb-488b-a504-29aa117c5139 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:17.013126 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:17.039046 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 313/627] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:17.049316 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed230f6-5838-4242-8bf9-ed8d91894896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:17.050347 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed230f6-5838-4242-8bf9-ed8d91894896 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:17.050865 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed230f6-5838-4242-8bf9-ed8d91894896 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:17.051375 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed230f6-5838-4242-8bf9-ed8d91894896 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:17.052317 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed230f6-5838-4242-8bf9-ed8d91894896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:17.052838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed230f6-5838-4242-8bf9-ed8d91894896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:17.054497 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed230f6-5838-4242-8bf9-ed8d91894896 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:17.070672 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 315/628] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:17.075592 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7716d0-2f54-44f5-a095-d1ac2eb5a105 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:17.076312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7716d0-2f54-44f5-a095-d1ac2eb5a105 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:17.076855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7716d0-2f54-44f5-a095-d1ac2eb5a105 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:17.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed230f6-5838-4242-8bf9-ed8d91894896 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:17.079171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 314/629] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:00:17.084196 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfdb72-3e8e-44c9-8c1f-2562cfd6afc4 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:17.084662 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfdb72-3e8e-44c9-8c1f-2562cfd6afc4 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:17.085099 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfdb72-3e8e-44c9-8c1f-2562cfd6afc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:17.116622 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-fbcfdb72-3e8e-44c9-8c1f-2562cfd6afc4 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:17.118388 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fbcfdb72-3e8e-44c9-8c1f-2562cfd6afc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:17.221923 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 315/630] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:00:17.239455 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed230f6-5838-4242-8bf9-ed8d91894896 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:17.281647 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5298f-7557-4c62-8158-8cd13aa6cf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:17.282564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5298f-7557-4c62-8158-8cd13aa6cf46 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:17.282993 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5298f-7557-4c62-8158-8cd13aa6cf46 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:17.283497 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5298f-7557-4c62-8158-8cd13aa6cf46 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:17.291257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5298f-7557-4c62-8158-8cd13aa6cf46 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:17.292905 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5298f-7557-4c62-8158-8cd13aa6cf46 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:17.295276 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-77c5298f-7557-4c62-8158-8cd13aa6cf46 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:17.306491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:17.307633 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:17.308815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:17.308815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:17.308815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:17.308815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:17.308815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:17.308815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:17.308815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:17.308815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:17.308815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:00:17.308815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:17.311888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 316/631] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 23 06:00:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:17.317797 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfdb72-3e8e-44c9-8c1f-2562cfd6afc4 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:17.389239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5298f-7557-4c62-8158-8cd13aa6cf46 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:17.488794 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfdb72-3e8e-44c9-8c1f-2562cfd6afc4 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:17.525397 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:17.532768 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5298f-7557-4c62-8158-8cd13aa6cf46 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:17.567851 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2165bcb-91b1-4431-9b24-5cd34900c80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:17.568480 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2165bcb-91b1-4431-9b24-5cd34900c80e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:17.568751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2165bcb-91b1-4431-9b24-5cd34900c80e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:17.569015 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2165bcb-91b1-4431-9b24-5cd34900c80e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:17.580401 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2165bcb-91b1-4431-9b24-5cd34900c80e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:17.585530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4bb90-4bfe-4b00-9eb4-73643712a107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:17.586143 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4bb90-4bfe-4b00-9eb4-73643712a107 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:17.586615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4bb90-4bfe-4b00-9eb4-73643712a107 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:17.586894 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4bb90-4bfe-4b00-9eb4-73643712a107 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-b2165bcb-91b1-4431-9b24-5cd34900c80e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:17.587247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:17.588136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:17.588970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:17.588970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:17.588970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:17.588970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:17.588970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:17.588970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:17.588970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:17.588970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:17.588970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:17.588970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:17.588970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:17.588970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:17.589599 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4bb90-4bfe-4b00-9eb4-73643712a107 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:17.589599 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4bb90-4bfe-4b00-9eb4-73643712a107 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:17.589728 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 317/632] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:17.589804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4bb90-4bfe-4b00-9eb4-73643712a107 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:17.594205 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2165bcb-91b1-4431-9b24-5cd34900c80e None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:17.613157 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 316/633] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 23 06:00:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:17.613157 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4bb90-4bfe-4b00-9eb4-73643712a107 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:17.674836 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:17.723729 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4bb90-4bfe-4b00-9eb4-73643712a107 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:17.739438 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2165bcb-91b1-4431-9b24-5cd34900c80e None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:17.773528 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb621e0f-623c-4eee-b300-fdf4b2260b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:17.774999 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb621e0f-623c-4eee-b300-fdf4b2260b0a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:17.775571 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb621e0f-623c-4eee-b300-fdf4b2260b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:17.776038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb621e0f-623c-4eee-b300-fdf4b2260b0a None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:17.776973 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb621e0f-623c-4eee-b300-fdf4b2260b0a None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:17.777566 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb621e0f-623c-4eee-b300-fdf4b2260b0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:17.779273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2623df-541e-4b06-9ff8-13b81964e8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:17.780012 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2623df-541e-4b06-9ff8-13b81964e8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:17.780223 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb621e0f-623c-4eee-b300-fdf4b2260b0a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:17.780373 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2623df-541e-4b06-9ff8-13b81964e8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:17.780789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2623df-541e-4b06-9ff8-13b81964e8b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:17.791251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 317/634] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:17.791838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2623df-541e-4b06-9ff8-13b81964e8b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:17.792580 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb621e0f-623c-4eee-b300-fdf4b2260b0a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-9c2623df-541e-4b06-9ff8-13b81964e8b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:17.798477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:17.799468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:17.800333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:17.800333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:17.800333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:17.800333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:17.800333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:17.800333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:17.800333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:17.800333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:17.800333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:17.800333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:17.800333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:17.800333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:17.804392 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 318/635] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:17.805338 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2623df-541e-4b06-9ff8-13b81964e8b5 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:17.841511 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:17.926765 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2623df-541e-4b06-9ff8-13b81964e8b5 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:17.940849 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb621e0f-623c-4eee-b300-fdf4b2260b0a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:17.945855 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:17.976179 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8994f-df6a-42c1-9d2e-1d67a4fc14d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:17.977179 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8994f-df6a-42c1-9d2e-1d67a4fc14d3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:17.977757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8994f-df6a-42c1-9d2e-1d67a4fc14d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:17.978248 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8994f-df6a-42c1-9d2e-1d67a4fc14d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:17.983416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59c0e3-1dea-4baa-8ccb-a80615c2e583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:17.984324 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59c0e3-1dea-4baa-8ccb-a80615c2e583 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:17.984822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59c0e3-1dea-4baa-8ccb-a80615c2e583 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:17.985306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59c0e3-1dea-4baa-8ccb-a80615c2e583 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:17.991802 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8994f-df6a-42c1-9d2e-1d67a4fc14d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:17.993462 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8994f-df6a-42c1-9d2e-1d67a4fc14d3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-3cc8994f-df6a-42c1-9d2e-1d67a4fc14d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:18.009003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:18.010008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:18.010930 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:18.010930 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:18.010930 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:18.010930 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 319/636] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:18.011163 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8994f-df6a-42c1-9d2e-1d67a4fc14d3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:18.075805 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59c0e3-1dea-4baa-8ccb-a80615c2e583 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:18.077619 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59c0e3-1dea-4baa-8ccb-a80615c2e583 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:18.110536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:18.127658 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8994f-df6a-42c1-9d2e-1d67a4fc14d3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:18.163901 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3939ebb-2ab9-48af-b6a7-048b9a203294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:18.164617 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3939ebb-2ab9-48af-b6a7-048b9a203294 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:18.164971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3939ebb-2ab9-48af-b6a7-048b9a203294 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:18.165355 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3939ebb-2ab9-48af-b6a7-048b9a203294 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:18.166057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3939ebb-2ab9-48af-b6a7-048b9a203294 None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:18.166351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3939ebb-2ab9-48af-b6a7-048b9a203294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:18.167742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3939ebb-2ab9-48af-b6a7-048b9a203294 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:18.177729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 320/637] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:18.179274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b7e3c-84c9-4ac9-bfc4-a71e3cac1166 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:18.179566 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b7e3c-84c9-4ac9-bfc4-a71e3cac1166 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:18.179837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b7e3c-84c9-4ac9-bfc4-a71e3cac1166 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:18.186243 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 321/638] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:18.186243 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b7e3c-84c9-4ac9-bfc4-a71e3cac1166 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:18.193834 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 318/639] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:00:18.198344 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59c0e3-1dea-4baa-8ccb-a80615c2e583 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:18.218365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:18.302675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b7e3c-84c9-4ac9-bfc4-a71e3cac1166 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:18.316536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59c0e3-1dea-4baa-8ccb-a80615c2e583 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:18.338156 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883f55c-e47c-4d6c-a4e0-07addf7337d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:18.338886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883f55c-e47c-4d6c-a4e0-07addf7337d8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:18.339164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883f55c-e47c-4d6c-a4e0-07addf7337d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:18.339439 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883f55c-e47c-4d6c-a4e0-07addf7337d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:18.340067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883f55c-e47c-4d6c-a4e0-07addf7337d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:18.340366 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883f55c-e47c-4d6c-a4e0-07addf7337d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:18.342541 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883f55c-e47c-4d6c-a4e0-07addf7337d8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:18.347991 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:18.353254 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc6a8e-433a-4b5d-89db-59e455a82e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:18.353885 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc6a8e-433a-4b5d-89db-59e455a82e58 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:18.354162 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc6a8e-433a-4b5d-89db-59e455a82e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:18.354434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc6a8e-433a-4b5d-89db-59e455a82e58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:18.357470 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 322/640] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:18.358960 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311d339-59fe-4133-8447-a9e13c4aa760 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:18.359240 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311d339-59fe-4133-8447-a9e13c4aa760 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:18.359510 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311d339-59fe-4133-8447-a9e13c4aa760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:18.371786 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc6a8e-433a-4b5d-89db-59e455a82e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:18.373804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc6a8e-433a-4b5d-89db-59e455a82e58 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:18.378794 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4311d339-59fe-4133-8447-a9e13c4aa760 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:18.380069 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4311d339-59fe-4133-8447-a9e13c4aa760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-83bc6a8e-433a-4b5d-89db-59e455a82e58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:18.390820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:18.391948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:18.398156 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:18.398156 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:18.398156 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:18.398156 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 319/641] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:18.398156 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc6a8e-433a-4b5d-89db-59e455a82e58 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:18.458753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 323/642] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:18.458753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311d339-59fe-4133-8447-a9e13c4aa760 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:18.491837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:18.527269 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc6a8e-433a-4b5d-89db-59e455a82e58 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:18.576660 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d170933-2a43-4e72-bbca-8bc3efbbd959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:18.577996 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d170933-2a43-4e72-bbca-8bc3efbbd959 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:18.578751 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d170933-2a43-4e72-bbca-8bc3efbbd959 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:18.579175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d170933-2a43-4e72-bbca-8bc3efbbd959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:18.580718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311d339-59fe-4133-8447-a9e13c4aa760 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:18.595692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d170933-2a43-4e72-bbca-8bc3efbbd959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:18.598173 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d170933-2a43-4e72-bbca-8bc3efbbd959 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-5d170933-2a43-4e72-bbca-8bc3efbbd959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:18.618363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:18.619608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:18.620691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:18.620691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:18.620691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:18.622126 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 320/643] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:18.623104 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d170933-2a43-4e72-bbca-8bc3efbbd959 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:18.624913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092b310-20ab-4cc0-841a-e0ff42c9eaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:18.625679 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092b310-20ab-4cc0-841a-e0ff42c9eaf3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:18.626094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092b310-20ab-4cc0-841a-e0ff42c9eaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:18.626463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092b310-20ab-4cc0-841a-e0ff42c9eaf3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:18.635299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092b310-20ab-4cc0-841a-e0ff42c9eaf3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:18.637100 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092b310-20ab-4cc0-841a-e0ff42c9eaf3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-0092b310-20ab-4cc0-841a-e0ff42c9eaf3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:18.652736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:18.654353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:18.655412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:18.655412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:18.655412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:18.655412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:18.655412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:18.655412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:18.655412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:18.655412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:18.655412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 06:00:18.655412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:18.657521 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 324/644] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:00:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:18.658497 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092b310-20ab-4cc0-841a-e0ff42c9eaf3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:18.671968 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:18.754158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:18.763720 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d170933-2a43-4e72-bbca-8bc3efbbd959 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:18.774301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092b310-20ab-4cc0-841a-e0ff42c9eaf3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:18.808524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfb0a9-9164-4aec-b064-2730586f921b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:18.809150 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfb0a9-9164-4aec-b064-2730586f921b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:18.809463 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfb0a9-9164-4aec-b064-2730586f921b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:18.809761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfb0a9-9164-4aec-b064-2730586f921b None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:18.810605 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acfb0a9-9164-4aec-b064-2730586f921b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:18.810890 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acfb0a9-9164-4aec-b064-2730586f921b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:18.812262 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acfb0a9-9164-4aec-b064-2730586f921b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:18.813359 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5437ec-7118-43ef-9010-1b2c75ad1d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:18.813961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5437ec-7118-43ef-9010-1b2c75ad1d90 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:18.814237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5437ec-7118-43ef-9010-1b2c75ad1d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:18.814532 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5437ec-7118-43ef-9010-1b2c75ad1d90 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:18.815239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5437ec-7118-43ef-9010-1b2c75ad1d90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:18.815539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5437ec-7118-43ef-9010-1b2c75ad1d90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:18.816895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5437ec-7118-43ef-9010-1b2c75ad1d90 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:18.825663 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 321/645] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:18.826788 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfb0a9-9164-4aec-b064-2730586f921b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:18.832842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 325/646] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:18.833676 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5437ec-7118-43ef-9010-1b2c75ad1d90 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:18.932296 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:18.939934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfb0a9-9164-4aec-b064-2730586f921b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:18.957036 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5437ec-7118-43ef-9010-1b2c75ad1d90 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:18.975878 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15828a6c-5471-40fc-90f1-c24d57ef6dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:18.976482 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15828a6c-5471-40fc-90f1-c24d57ef6dda None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:18.976760 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15828a6c-5471-40fc-90f1-c24d57ef6dda None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:18.977024 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15828a6c-5471-40fc-90f1-c24d57ef6dda None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:18.977754 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15828a6c-5471-40fc-90f1-c24d57ef6dda None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:18.978049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15828a6c-5471-40fc-90f1-c24d57ef6dda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:18.979569 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15828a6c-5471-40fc-90f1-c24d57ef6dda None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:18.994993 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3b5a7-e856-429f-acbd-221754091a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:18.995895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3b5a7-e856-429f-acbd-221754091a3b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:18.996172 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3b5a7-e856-429f-acbd-221754091a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:18.996461 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3b5a7-e856-429f-acbd-221754091a3b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:18.998978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 322/647] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 23 06:00:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:19.000044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15828a6c-5471-40fc-90f1-c24d57ef6dda None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:19.011965 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3b5a7-e856-429f-acbd-221754091a3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-49c3b5a7-e856-429f-acbd-221754091a3b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:19.016713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:19.017678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:19.018500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:19.018500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:19.018500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:19.018500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:19.018500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:19.018500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:19.018500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:19.018500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:19.018500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:19.018500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:19.018500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:19.018500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:19.018500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 326/648] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:19.019241 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3b5a7-e856-429f-acbd-221754091a3b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:19.025609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:19.119526 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15828a6c-5471-40fc-90f1-c24d57ef6dda None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:19.133619 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3b5a7-e856-429f-acbd-221754091a3b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:19.152770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ee630-0957-4cbe-9bb8-6bd9c33f0836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:19.154005 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ee630-0957-4cbe-9bb8-6bd9c33f0836 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:19.154429 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ee630-0957-4cbe-9bb8-6bd9c33f0836 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:19.154739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ee630-0957-4cbe-9bb8-6bd9c33f0836 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:19.161659 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ee630-0957-4cbe-9bb8-6bd9c33f0836 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:19.163134 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ee630-0957-4cbe-9bb8-6bd9c33f0836 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:19.167977 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f9f3c-1500-4642-9e67-e137cc1c02db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:19.168618 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f9f3c-1500-4642-9e67-e137cc1c02db None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:19.169434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f9f3c-1500-4642-9e67-e137cc1c02db None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:19.169701 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f9f3c-1500-4642-9e67-e137cc1c02db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-6b8ee630-0957-4cbe-9bb8-6bd9c33f0836 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:19.174955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:19.175986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:19.176944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:19.176944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:19.176944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:19.176944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:19.176944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:19.176944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:19.176944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:19.176944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:19.176944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 06:00:19.176944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:19.176944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 323/649] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:00:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:19.177558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f9f3c-1500-4642-9e67-e137cc1c02db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:19.177629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ee630-0957-4cbe-9bb8-6bd9c33f0836 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:19.177737 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f9f3c-1500-4642-9e67-e137cc1c02db None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:19.193130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-1c0f9f3c-1500-4642-9e67-e137cc1c02db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:19.193130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:19.193130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:19.193130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:19.193130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:19.193130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:19.193130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:19.193130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:19.193130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:19.193130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:19.193130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:19.193130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:19.193130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:19.193130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:19.193130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:19.194123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:19.195194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:19.195194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:19.195194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:19.195194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:19.195194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:19.195194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:19.195194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:19.195194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:19.195194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 327/650] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:19.195194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f9f3c-1500-4642-9e67-e137cc1c02db None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:19.197523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:19.260895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:19.285973 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ee630-0957-4cbe-9bb8-6bd9c33f0836 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:19.305132 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f9f3c-1500-4642-9e67-e137cc1c02db None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:19.322112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd65c1-4aed-42e8-b505-7c6a0f68ddba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:19.322738 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd65c1-4aed-42e8-b505-7c6a0f68ddba None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:19.323015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd65c1-4aed-42e8-b505-7c6a0f68ddba None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:19.323282 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd65c1-4aed-42e8-b505-7c6a0f68ddba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:19.338873 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c11e35-c720-49c0-95b2-17283a5fb0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:19.339475 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c11e35-c720-49c0-95b2-17283a5fb0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:19.339757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c11e35-c720-49c0-95b2-17283a5fb0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:19.340032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c11e35-c720-49c0-95b2-17283a5fb0a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:19.341473 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c11e35-c720-49c0-95b2-17283a5fb0a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:19.341839 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c11e35-c720-49c0-95b2-17283a5fb0a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:19.343164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c11e35-c720-49c0-95b2-17283a5fb0a6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:19.351529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 328/651] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:19.352729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c67f6-3956-4ff5-92a1-3982b17f00f6 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:19.353016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c67f6-3956-4ff5-92a1-3982b17f00f6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:19.353292 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c67f6-3956-4ff5-92a1-3982b17f00f6 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:19.354555 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 329/652] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:19.355215 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c67f6-3956-4ff5-92a1-3982b17f00f6 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:19.393566 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd65c1-4aed-42e8-b505-7c6a0f68ddba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:19.394984 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd65c1-4aed-42e8-b505-7c6a0f68ddba None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:19.409646 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:19.469593 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c67f6-3956-4ff5-92a1-3982b17f00f6 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:19.493945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:19.500916 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 324/653] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:00:19.501793 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd65c1-4aed-42e8-b505-7c6a0f68ddba None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:19.510656 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a09618-dd86-4222-b1b2-3a894e9a33ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:19.511275 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a09618-dd86-4222-b1b2-3a894e9a33ec None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:19.511555 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a09618-dd86-4222-b1b2-3a894e9a33ec None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:19.511827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a09618-dd86-4222-b1b2-3a894e9a33ec None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:19.513108 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a09618-dd86-4222-b1b2-3a894e9a33ec None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:19.513537 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a09618-dd86-4222-b1b2-3a894e9a33ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:19.515020 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a09618-dd86-4222-b1b2-3a894e9a33ec None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:19.534017 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 330/654] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Aug 23 06:00:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:19.536187 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16065722-1fc6-4c56-96b1-a95fb53169b0 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:19.536649 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16065722-1fc6-4c56-96b1-a95fb53169b0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:19.536926 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16065722-1fc6-4c56-96b1-a95fb53169b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:19.555536 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-16065722-1fc6-4c56-96b1-a95fb53169b0 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:19.556801 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-16065722-1fc6-4c56-96b1-a95fb53169b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:19.610783 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd65c1-4aed-42e8-b505-7c6a0f68ddba None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:19.626014 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:19.629707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 331/655] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:19.630650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16065722-1fc6-4c56-96b1-a95fb53169b0 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:19.650254 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b918c32-feda-4e62-b7f2-59cac71d5b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:19.650826 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b918c32-feda-4e62-b7f2-59cac71d5b92 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:19.651105 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b918c32-feda-4e62-b7f2-59cac71d5b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:19.651376 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b918c32-feda-4e62-b7f2-59cac71d5b92 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:19.652061 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b918c32-feda-4e62-b7f2-59cac71d5b92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:19.652363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b918c32-feda-4e62-b7f2-59cac71d5b92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:19.654678 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b918c32-feda-4e62-b7f2-59cac71d5b92 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:19.667603 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 325/656] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:19.673773 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b918c32-feda-4e62-b7f2-59cac71d5b92 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:19.749521 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16065722-1fc6-4c56-96b1-a95fb53169b0 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:19.767387 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:19.787952 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b918c32-feda-4e62-b7f2-59cac71d5b92 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:19.789596 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df3db7-a4cb-4108-a53e-df1d7b49f250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:19.790865 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df3db7-a4cb-4108-a53e-df1d7b49f250 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:19.790865 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df3db7-a4cb-4108-a53e-df1d7b49f250 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:19.790865 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df3db7-a4cb-4108-a53e-df1d7b49f250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:19.805056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df3db7-a4cb-4108-a53e-df1d7b49f250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:19.806992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df3db7-a4cb-4108-a53e-df1d7b49f250 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-98df3db7-a4cb-4108-a53e-df1d7b49f250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:19.826458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:19.827862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:19.829037 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:19.829037 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:19.829037 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:19.829251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca7ae9-1375-45fb-941c-44dd51abee70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:19.829251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca7ae9-1375-45fb-941c-44dd51abee70 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:19.829573 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca7ae9-1375-45fb-941c-44dd51abee70 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:19.829909 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca7ae9-1375-45fb-941c-44dd51abee70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:19.834635 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 332/657] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:19.834635 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df3db7-a4cb-4108-a53e-df1d7b49f250 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:19.846642 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca7ae9-1375-45fb-941c-44dd51abee70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:19.848247 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca7ae9-1375-45fb-941c-44dd51abee70 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-0bca7ae9-1375-45fb-941c-44dd51abee70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:19.863562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:19.864790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:19.865777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:19.865777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:19.865777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:19.869645 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 326/658] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:19.870376 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca7ae9-1375-45fb-941c-44dd51abee70 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:19.900902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:19.945186 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df3db7-a4cb-4108-a53e-df1d7b49f250 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:19.977493 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca7ae9-1375-45fb-941c-44dd51abee70 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:19.979758 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d045ac-aad2-4869-9926-5e972635d9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:19.980360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d045ac-aad2-4869-9926-5e972635d9ba None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:19.980638 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d045ac-aad2-4869-9926-5e972635d9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:19.980904 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d045ac-aad2-4869-9926-5e972635d9ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:20.012210 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:20.014697 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b99a4e-9fcb-4125-9791-44ed676064a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:20.015299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b99a4e-9fcb-4125-9791-44ed676064a7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:20.015886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b99a4e-9fcb-4125-9791-44ed676064a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:20.016157 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b99a4e-9fcb-4125-9791-44ed676064a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:20.016882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b99a4e-9fcb-4125-9791-44ed676064a7 None admin] RBAC: Adding query filter params (role.id=3209a3d044b84a0ebd7cf1edf2da7120, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=91267a049343452eb208205ee3ea27c4) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:20.017188 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b99a4e-9fcb-4125-9791-44ed676064a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:20.018693 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b99a4e-9fcb-4125-9791-44ed676064a7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:20.024439 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 327/659] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:19 2022] GET /identity/v3/role_assignments?user.id=91267a049343452eb208205ee3ea27c4&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=3209a3d044b84a0ebd7cf1edf2da7120 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:20.025239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b99a4e-9fcb-4125-9791-44ed676064a7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:20.055818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d045ac-aad2-4869-9926-5e972635d9ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:20.057292 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d045ac-aad2-4869-9926-5e972635d9ba None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:20.135735 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:20.148275 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b99a4e-9fcb-4125-9791-44ed676064a7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:20.173245 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 333/660] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:00:20.173746 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d045ac-aad2-4869-9926-5e972635d9ba None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:20.197835 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0cafc-5821-41dd-a8cc-cdebd44ea769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:20.198607 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0cafc-5821-41dd-a8cc-cdebd44ea769 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:20.198861 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0cafc-5821-41dd-a8cc-cdebd44ea769 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:20.199133 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0cafc-5821-41dd-a8cc-cdebd44ea769 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:20.201249 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0cafc-5821-41dd-a8cc-cdebd44ea769 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:20.201249 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0cafc-5821-41dd-a8cc-cdebd44ea769 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:20.202437 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0cafc-5821-41dd-a8cc-cdebd44ea769 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:20.221461 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 328/661] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:20.222601 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0cafc-5821-41dd-a8cc-cdebd44ea769 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:20.257615 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:20.301232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d045ac-aad2-4869-9926-5e972635d9ba None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:20.339882 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b751c-ac06-4692-b986-141b625296d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:20.340980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b751c-ac06-4692-b986-141b625296d8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:20.341269 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b751c-ac06-4692-b986-141b625296d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:20.341537 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b751c-ac06-4692-b986-141b625296d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:20.342170 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b751c-ac06-4692-b986-141b625296d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:20.342461 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b751c-ac06-4692-b986-141b625296d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:20.343848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b751c-ac06-4692-b986-141b625296d8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:20.345816 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0cafc-5821-41dd-a8cc-cdebd44ea769 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:20.355195 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 334/662] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:20.356013 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b751c-ac06-4692-b986-141b625296d8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:20.386799 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014c8e1-a4aa-4122-a5e8-97f1c8bce3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:20.387776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014c8e1-a4aa-4122-a5e8-97f1c8bce3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:20.388201 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014c8e1-a4aa-4122-a5e8-97f1c8bce3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:20.388620 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014c8e1-a4aa-4122-a5e8-97f1c8bce3d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:20.401482 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014c8e1-a4aa-4122-a5e8-97f1c8bce3d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-c014c8e1-a4aa-4122-a5e8-97f1c8bce3d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:20.408899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:20.410010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:20.411052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:20.411052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:20.411052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:20.411052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:20.411052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:20.411052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:20.411052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:20.411052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:20.411052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:20.411052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:20.411052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:20.411052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:20.411946 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 329/663] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:20.412541 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014c8e1-a4aa-4122-a5e8-97f1c8bce3d0 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:20.428679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:20.471817 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b751c-ac06-4692-b986-141b625296d8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:20.498217 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:20.510945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2df38-9da0-478b-b9e3-bc82a3b7d499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:20.511833 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2df38-9da0-478b-b9e3-bc82a3b7d499 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:20.512119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2df38-9da0-478b-b9e3-bc82a3b7d499 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:20.512544 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2df38-9da0-478b-b9e3-bc82a3b7d499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:20.524232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2df38-9da0-478b-b9e3-bc82a3b7d499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:20.525689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2df38-9da0-478b-b9e3-bc82a3b7d499 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:20.532577 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014c8e1-a4aa-4122-a5e8-97f1c8bce3d0 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-56d2df38-9da0-478b-b9e3-bc82a3b7d499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:20.540999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:20.542091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:20.543012 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:20.543012 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:20.543012 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:20.543012 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 335/664] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:20.543270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2df38-9da0-478b-b9e3-bc82a3b7d499 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:20.571903 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345aa1a7-9a33-4125-9b21-ca312b37fce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:20.572526 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345aa1a7-9a33-4125-9b21-ca312b37fce3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:20.572801 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345aa1a7-9a33-4125-9b21-ca312b37fce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:20.573068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345aa1a7-9a33-4125-9b21-ca312b37fce3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:20.580964 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345aa1a7-9a33-4125-9b21-ca312b37fce3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:20.582459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345aa1a7-9a33-4125-9b21-ca312b37fce3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-345aa1a7-9a33-4125-9b21-ca312b37fce3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:20.597112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:20.598277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:20.599489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:20.599489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:20.599489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:20.599489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:20.599489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:20.599489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:20.599489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:20.599489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:20.599489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 06:00:20.599489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:20.605058 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 330/665] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:20.605766 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345aa1a7-9a33-4125-9b21-ca312b37fce3 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:20.660837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2df38-9da0-478b-b9e3-bc82a3b7d499 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:20.679276 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:20.704929 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb7778-81af-4cb4-8825-6e87a2ff5c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:20.705573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb7778-81af-4cb4-8825-6e87a2ff5c59 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:20.706100 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb7778-81af-4cb4-8825-6e87a2ff5c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:20.706431 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb7778-81af-4cb4-8825-6e87a2ff5c59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:20.712494 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345aa1a7-9a33-4125-9b21-ca312b37fce3 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:20.714862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb7778-81af-4cb4-8825-6e87a2ff5c59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:20.716413 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb7778-81af-4cb4-8825-6e87a2ff5c59 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:20.728721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-8cfb7778-81af-4cb4-8825-6e87a2ff5c59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:20.728721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:20.728721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:20.728721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:20.728721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:20.728721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:20.728721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:20.728721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:20.728721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:20.728721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:20.728721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:20.728721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:20.728721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:20.728721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:20.728721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:20.729675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:20.731644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:20.731644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:20.731644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:20.731644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:20.731644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:20.731644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:20.731644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:20.731644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:20.731644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 336/666] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:20.731644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb7778-81af-4cb4-8825-6e87a2ff5c59 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:20.754451 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9219c-eb0e-40cd-b070-0062aad08ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:20.755081 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9219c-eb0e-40cd-b070-0062aad08ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:20.755361 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9219c-eb0e-40cd-b070-0062aad08ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:20.755634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9219c-eb0e-40cd-b070-0062aad08ff5 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:20.756295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9219c-eb0e-40cd-b070-0062aad08ff5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:20.756596 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9219c-eb0e-40cd-b070-0062aad08ff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:20.757969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9219c-eb0e-40cd-b070-0062aad08ff5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:20.766833 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 331/667] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:20.767651 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9219c-eb0e-40cd-b070-0062aad08ff5 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:20.782854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:20.852439 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb7778-81af-4cb4-8825-6e87a2ff5c59 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:20.877596 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9219c-eb0e-40cd-b070-0062aad08ff5 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:20.888604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c2de-fb63-4747-baf3-d86ebe2014a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:20.889302 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c2de-fb63-4747-baf3-d86ebe2014a0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:20.889629 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c2de-fb63-4747-baf3-d86ebe2014a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:20.889901 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c2de-fb63-4747-baf3-d86ebe2014a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:20.890578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01c2de-fb63-4747-baf3-d86ebe2014a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:20.890883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01c2de-fb63-4747-baf3-d86ebe2014a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:20.892262 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01c2de-fb63-4747-baf3-d86ebe2014a0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:20.906262 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 337/668] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:20.907786 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c2de-fb63-4747-baf3-d86ebe2014a0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:20.915297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:20.922568 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf780811-f861-4bf7-bd81-3483dc17e0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:20.923247 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf780811-f861-4bf7-bd81-3483dc17e0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:20.923590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf780811-f861-4bf7-bd81-3483dc17e0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:20.923932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf780811-f861-4bf7-bd81-3483dc17e0e2 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:20.924744 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf780811-f861-4bf7-bd81-3483dc17e0e2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:20.925046 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf780811-f861-4bf7-bd81-3483dc17e0e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:20.926500 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf780811-f861-4bf7-bd81-3483dc17e0e2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:20.946709 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 332/669] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Aug 23 06:00:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:20.946709 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf780811-f861-4bf7-bd81-3483dc17e0e2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:21.012111 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:21.019761 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c2de-fb63-4747-baf3-d86ebe2014a0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:21.058109 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2ce04-a4cb-4b60-af89-ecb467e1ab08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:21.058962 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2ce04-a4cb-4b60-af89-ecb467e1ab08 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:21.059975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2ce04-a4cb-4b60-af89-ecb467e1ab08 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:21.059975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2ce04-a4cb-4b60-af89-ecb467e1ab08 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:21.060920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2ce04-a4cb-4b60-af89-ecb467e1ab08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:21.061090 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2ce04-a4cb-4b60-af89-ecb467e1ab08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:21.062441 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2ce04-a4cb-4b60-af89-ecb467e1ab08 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:21.064743 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf780811-f861-4bf7-bd81-3483dc17e0e2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:21.075508 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 338/670] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:21.076359 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2ce04-a4cb-4b60-af89-ecb467e1ab08 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:21.114866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4d531-7afe-4d69-8d9d-7e061d27a6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:21.115764 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4d531-7afe-4d69-8d9d-7e061d27a6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:21.116039 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4d531-7afe-4d69-8d9d-7e061d27a6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:21.116330 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4d531-7afe-4d69-8d9d-7e061d27a6b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:21.132026 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4d531-7afe-4d69-8d9d-7e061d27a6b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:21.134217 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4d531-7afe-4d69-8d9d-7e061d27a6b4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-13d4d531-7afe-4d69-8d9d-7e061d27a6b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:21.152140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:21.153557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:21.154836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:21.154836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:21.154836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:21.161026 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 333/671] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:21.161026 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4d531-7afe-4d69-8d9d-7e061d27a6b4 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:21.220123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2ce04-a4cb-4b60-af89-ecb467e1ab08 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:21.227703 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:21.259824 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfa482-bd93-4ef3-af19-af6173b65011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:21.260615 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfa482-bd93-4ef3-af19-af6173b65011 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:21.260987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfa482-bd93-4ef3-af19-af6173b65011 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:21.261334 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfa482-bd93-4ef3-af19-af6173b65011 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:21.270299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfa482-bd93-4ef3-af19-af6173b65011 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:21.271925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfa482-bd93-4ef3-af19-af6173b65011 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:21.283954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-66bfa482-bd93-4ef3-af19-af6173b65011 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:21.283954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:21.283954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:21.283954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:21.283954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:21.283954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:21.283954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:21.283954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:21.283954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:21.283954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:21.283954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:21.283954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:21.283954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:21.283954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:21.283954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:21.285061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:21.286089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:21.286089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:21.286089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:21.286089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:21.286089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:21.286089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:21.286089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:21.286089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:21.286089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 339/672] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:21.290882 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfa482-bd93-4ef3-af19-af6173b65011 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:21.300877 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4d531-7afe-4d69-8d9d-7e061d27a6b4 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:21.316041 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:21.341021 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285c048-70f3-4734-b36d-28ffe19254ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:21.341848 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285c048-70f3-4734-b36d-28ffe19254ef None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:21.342122 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285c048-70f3-4734-b36d-28ffe19254ef None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:21.342389 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285c048-70f3-4734-b36d-28ffe19254ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:21.411670 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfa482-bd93-4ef3-af19-af6173b65011 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:21.415317 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285c048-70f3-4734-b36d-28ffe19254ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:21.416775 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285c048-70f3-4734-b36d-28ffe19254ef None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:21.450442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979a8ba-2b37-4f44-86bc-3f4f28755a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:21.451440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979a8ba-2b37-4f44-86bc-3f4f28755a46 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:21.451732 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979a8ba-2b37-4f44-86bc-3f4f28755a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:21.452010 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979a8ba-2b37-4f44-86bc-3f4f28755a46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:21.452741 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979a8ba-2b37-4f44-86bc-3f4f28755a46 None admin] RBAC: Adding query filter params (role.id=3209a3d044b84a0ebd7cf1edf2da7120, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=89621d839d1c44798f691db4d49e1278) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:21.453051 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979a8ba-2b37-4f44-86bc-3f4f28755a46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:21.454770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979a8ba-2b37-4f44-86bc-3f4f28755a46 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:21.461638 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 340/673] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:21 2022] GET /identity/v3/role_assignments?user.id=89621d839d1c44798f691db4d49e1278&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=3209a3d044b84a0ebd7cf1edf2da7120 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:21.462470 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979a8ba-2b37-4f44-86bc-3f4f28755a46 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:21.463074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:21.546872 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 334/674] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:00:21.547888 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285c048-70f3-4734-b36d-28ffe19254ef None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:21.579188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:21.602983 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979a8ba-2b37-4f44-86bc-3f4f28755a46 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:21.646401 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f983c-6e75-4599-9a7b-b9419b6fe132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:21.647053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f983c-6e75-4599-9a7b-b9419b6fe132 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:21.647332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f983c-6e75-4599-9a7b-b9419b6fe132 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:21.647601 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f983c-6e75-4599-9a7b-b9419b6fe132 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:21.648245 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f983c-6e75-4599-9a7b-b9419b6fe132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:21.648549 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f983c-6e75-4599-9a7b-b9419b6fe132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:21.649941 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f983c-6e75-4599-9a7b-b9419b6fe132 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:21.667012 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 341/675] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:21.667698 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f983c-6e75-4599-9a7b-b9419b6fe132 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:21.692480 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285c048-70f3-4734-b36d-28ffe19254ef None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:21.744507 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae0b71-9a83-4f7d-b357-3c606c4e7403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:21.745222 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae0b71-9a83-4f7d-b357-3c606c4e7403 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:21.745511 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae0b71-9a83-4f7d-b357-3c606c4e7403 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:21.745782 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae0b71-9a83-4f7d-b357-3c606c4e7403 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:21.746717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae0b71-9a83-4f7d-b357-3c606c4e7403 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:21.747322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae0b71-9a83-4f7d-b357-3c606c4e7403 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:21.749060 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae0b71-9a83-4f7d-b357-3c606c4e7403 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:21.761589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 335/676] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:21.762970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae0b71-9a83-4f7d-b357-3c606c4e7403 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:21.768221 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:21.801068 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f983c-6e75-4599-9a7b-b9419b6fe132 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:21.838948 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abbf4d-f937-4e9b-8729-456214663598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:21.839576 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abbf4d-f937-4e9b-8729-456214663598 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:21.839851 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abbf4d-f937-4e9b-8729-456214663598 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:21.840219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abbf4d-f937-4e9b-8729-456214663598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:21.852184 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:21.855087 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abbf4d-f937-4e9b-8729-456214663598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:21.857146 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abbf4d-f937-4e9b-8729-456214663598 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-82abbf4d-f937-4e9b-8729-456214663598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:21.875112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:21.876291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:21.877192 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:21.877192 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:21.877192 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:21.877656 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 342/677] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:21.878846 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abbf4d-f937-4e9b-8729-456214663598 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:21.886701 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae0b71-9a83-4f7d-b357-3c606c4e7403 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:21.927692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c5df5-2f15-479b-b57a-f9989a4fee70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:21.928336 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c5df5-2f15-479b-b57a-f9989a4fee70 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:21.928612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c5df5-2f15-479b-b57a-f9989a4fee70 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:21.928985 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c5df5-2f15-479b-b57a-f9989a4fee70 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:21.938264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c5df5-2f15-479b-b57a-f9989a4fee70 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:21.939956 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c5df5-2f15-479b-b57a-f9989a4fee70 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-ef1c5df5-2f15-479b-b57a-f9989a4fee70 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:21.953501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:21.955707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:21.957851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:21.957851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:21.957851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:21.957851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:21.957851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:21.957851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:21.957851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:21.957851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:21.957851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:00:21.957851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:21.957851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 336/678] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:00:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:21.957851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c5df5-2f15-479b-b57a-f9989a4fee70 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:21.993932 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abbf4d-f937-4e9b-8729-456214663598 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:22.028464 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:22.032728 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8032372-ef4b-4cd9-b802-a8158e92fec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:22.034107 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8032372-ef4b-4cd9-b802-a8158e92fec6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:22.034560 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8032372-ef4b-4cd9-b802-a8158e92fec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:22.035373 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8032372-ef4b-4cd9-b802-a8158e92fec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:22.036250 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8032372-ef4b-4cd9-b802-a8158e92fec6 None admin] RBAC: Adding query filter params (role.id=3209a3d044b84a0ebd7cf1edf2da7120, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=c741b92684794c338d63913774ab841a) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:22.036728 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8032372-ef4b-4cd9-b802-a8158e92fec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:22.038430 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8032372-ef4b-4cd9-b802-a8158e92fec6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:22.050663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 343/679] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:21 2022] GET /identity/v3/role_assignments?user.id=c741b92684794c338d63913774ab841a&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=3209a3d044b84a0ebd7cf1edf2da7120 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:22.052047 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8032372-ef4b-4cd9-b802-a8158e92fec6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:22.073860 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c5df5-2f15-479b-b57a-f9989a4fee70 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:22.117453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcbcf8-e82d-4397-9939-0aef3cd1fe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:22.118381 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcbcf8-e82d-4397-9939-0aef3cd1fe31 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:22.118712 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcbcf8-e82d-4397-9939-0aef3cd1fe31 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:22.118986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcbcf8-e82d-4397-9939-0aef3cd1fe31 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:22.119851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcbcf8-e82d-4397-9939-0aef3cd1fe31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:22.120160 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcbcf8-e82d-4397-9939-0aef3cd1fe31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:22.123013 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcbcf8-e82d-4397-9939-0aef3cd1fe31 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:22.132966 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:22.140576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 337/680] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:22.142246 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc1f48-11f4-4f4a-b454-8a3e9e017e05 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:22.142751 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc1f48-11f4-4f4a-b454-8a3e9e017e05 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:22.143193 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc1f48-11f4-4f4a-b454-8a3e9e017e05 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:22.145120 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 338/681] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:22.146122 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc1f48-11f4-4f4a-b454-8a3e9e017e05 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:22.183573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8032372-ef4b-4cd9-b802-a8158e92fec6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:22.220894 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0ec3c-f878-4987-a91e-573615837711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:22.221518 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0ec3c-f878-4987-a91e-573615837711 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:22.221792 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0ec3c-f878-4987-a91e-573615837711 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:22.222061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0ec3c-f878-4987-a91e-573615837711 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:22.222850 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0ec3c-f878-4987-a91e-573615837711 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:22.223153 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0ec3c-f878-4987-a91e-573615837711 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:22.224605 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0ec3c-f878-4987-a91e-573615837711 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:22.250208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 344/682] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 23 06:00:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:22.250208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da76baf-cd01-475e-b9ee-40cb500fb813 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:22.250208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da76baf-cd01-475e-b9ee-40cb500fb813 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:22.250208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da76baf-cd01-475e-b9ee-40cb500fb813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:22.267290 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8da76baf-cd01-475e-b9ee-40cb500fb813 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:22.268481 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8da76baf-cd01-475e-b9ee-40cb500fb813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:22.281024 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc1f48-11f4-4f4a-b454-8a3e9e017e05 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:22.308719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:22.327804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08bac1c-3d00-4bdd-b443-dca8d0686ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:22.328456 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08bac1c-3d00-4bdd-b443-dca8d0686ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:22.328732 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08bac1c-3d00-4bdd-b443-dca8d0686ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:22.329005 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08bac1c-3d00-4bdd-b443-dca8d0686ca1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:22.336655 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08bac1c-3d00-4bdd-b443-dca8d0686ca1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:22.338114 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08bac1c-3d00-4bdd-b443-dca8d0686ca1 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:22.352312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-b08bac1c-3d00-4bdd-b443-dca8d0686ca1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:22.352312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:22.352312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:22.352312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:22.352312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:22.352312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:22.352312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:22.352312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:22.352312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:22.352312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:22.352312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:22.352312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:22.352312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:22.352312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:22.352312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:22.353416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:22.354570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:22.354570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:22.354570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:22.354570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:22.354570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:22.354570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:22.354570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:22.354570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:22.355234 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 345/683] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:22.357200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 339/684] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:22.357322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da76baf-cd01-475e-b9ee-40cb500fb813 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:22.363654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08bac1c-3d00-4bdd-b443-dca8d0686ca1 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:22.415987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:22.502443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08bac1c-3d00-4bdd-b443-dca8d0686ca1 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:22.506698 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da76baf-cd01-475e-b9ee-40cb500fb813 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:22.578361 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e46690-ab8c-453a-82e6-18db4b5279a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:22.578918 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e46690-ab8c-453a-82e6-18db4b5279a3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:22.579201 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e46690-ab8c-453a-82e6-18db4b5279a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:22.579476 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e46690-ab8c-453a-82e6-18db4b5279a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:22.580179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e46690-ab8c-453a-82e6-18db4b5279a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:22.580486 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e46690-ab8c-453a-82e6-18db4b5279a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:22.581928 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e46690-ab8c-453a-82e6-18db4b5279a3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:22.584255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3fe88-5c61-4599-b0e7-46f2007592fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:22.584868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3fe88-5c61-4599-b0e7-46f2007592fb None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:22.585144 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3fe88-5c61-4599-b0e7-46f2007592fb None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:22.585417 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3fe88-5c61-4599-b0e7-46f2007592fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:22.613892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 340/685] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:22.618695 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e46690-ab8c-453a-82e6-18db4b5279a3 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:22.686760 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:22.688683 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3fe88-5c61-4599-b0e7-46f2007592fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:22.690312 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3fe88-5c61-4599-b0e7-46f2007592fb None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:22.737969 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:22.752846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e46690-ab8c-453a-82e6-18db4b5279a3 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:22.790729 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fed52-c2f2-4e77-87d2-1a29abbed9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:22.791336 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fed52-c2f2-4e77-87d2-1a29abbed9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:22.791711 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fed52-c2f2-4e77-87d2-1a29abbed9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:22.791983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fed52-c2f2-4e77-87d2-1a29abbed9e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:22.792830 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fed52-c2f2-4e77-87d2-1a29abbed9e9 None admin] RBAC: Adding query filter params (role.id=3209a3d044b84a0ebd7cf1edf2da7120, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=dae9eff3bbf04496af2300a24c630eb5) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:22.793134 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fed52-c2f2-4e77-87d2-1a29abbed9e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:22.794738 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fed52-c2f2-4e77-87d2-1a29abbed9e9 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:22.796333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 346/686] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:00:22.797026 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3fe88-5c61-4599-b0e7-46f2007592fb None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:22.802027 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 341/687] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:22 2022] GET /identity/v3/role_assignments?user.id=dae9eff3bbf04496af2300a24c630eb5&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=3209a3d044b84a0ebd7cf1edf2da7120 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:22.803263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fed52-c2f2-4e77-87d2-1a29abbed9e9 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:22.920499 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3fe88-5c61-4599-b0e7-46f2007592fb None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:22.927082 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fed52-c2f2-4e77-87d2-1a29abbed9e9 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:22.935954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:22.960333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909ce89-78cd-43bf-a471-fe521807e58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:22.961102 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909ce89-78cd-43bf-a471-fe521807e58f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:22.961343 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909ce89-78cd-43bf-a471-fe521807e58f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:22.961604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909ce89-78cd-43bf-a471-fe521807e58f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:22.971924 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222cd93-f8d0-48d7-af26-6c7121019732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:22.972917 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222cd93-f8d0-48d7-af26-6c7121019732 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:22.973395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222cd93-f8d0-48d7-af26-6c7121019732 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:22.973862 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222cd93-f8d0-48d7-af26-6c7121019732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:22.974069 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909ce89-78cd-43bf-a471-fe521807e58f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-3909ce89-78cd-43bf-a471-fe521807e58f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:22.979025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:22.979975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:22.980818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:22.980818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:22.980818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:22.980818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:22.980818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:22.980818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:22.980818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:22.980818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:22.980818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:22.980818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:22.980818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:00:22.980818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:22.980818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 347/688] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:00:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:22.986487 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909ce89-78cd-43bf-a471-fe521807e58f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:22.989742 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222cd93-f8d0-48d7-af26-6c7121019732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:22.991171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:22.993402 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222cd93-f8d0-48d7-af26-6c7121019732 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-8222cd93-f8d0-48d7-af26-6c7121019732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:23.011910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:23.013725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:23.015090 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:23.015090 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:23.015090 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:23.016955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 342/689] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:23.027521 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222cd93-f8d0-48d7-af26-6c7121019732 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:23.128345 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909ce89-78cd-43bf-a471-fe521807e58f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:23.167640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222cd93-f8d0-48d7-af26-6c7121019732 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:23.174564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ebdfb-f2f2-45bb-9c4c-dd719bb33f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:23.175116 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ebdfb-f2f2-45bb-9c4c-dd719bb33f15 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:23.175394 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ebdfb-f2f2-45bb-9c4c-dd719bb33f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:23.175663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ebdfb-f2f2-45bb-9c4c-dd719bb33f15 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:23.176330 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ebdfb-f2f2-45bb-9c4c-dd719bb33f15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:23.176628 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ebdfb-f2f2-45bb-9c4c-dd719bb33f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:23.178693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ebdfb-f2f2-45bb-9c4c-dd719bb33f15 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:23.190781 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 348/690] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:00:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:23.196155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ebdfb-f2f2-45bb-9c4c-dd719bb33f15 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:23.210842 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc5bce-d2aa-4c0e-954b-9a3ea0ef1957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:23.212082 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc5bce-d2aa-4c0e-954b-9a3ea0ef1957 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:23.212919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc5bce-d2aa-4c0e-954b-9a3ea0ef1957 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:23.213404 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc5bce-d2aa-4c0e-954b-9a3ea0ef1957 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:23.214404 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc5bce-d2aa-4c0e-954b-9a3ea0ef1957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:23.214919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc5bce-d2aa-4c0e-954b-9a3ea0ef1957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:23.217562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc5bce-d2aa-4c0e-954b-9a3ea0ef1957 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:23.234728 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 343/691] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:23.249086 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc5bce-d2aa-4c0e-954b-9a3ea0ef1957 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:23.274632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:23.313150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:23.386192 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ebdfb-f2f2-45bb-9c4c-dd719bb33f15 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:23.408148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc5bce-d2aa-4c0e-954b-9a3ea0ef1957 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:23.448606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30f90a-a11a-4e65-a09e-46091a1e2995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:23.449176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30f90a-a11a-4e65-a09e-46091a1e2995 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:23.449456 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30f90a-a11a-4e65-a09e-46091a1e2995 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:23.449756 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30f90a-a11a-4e65-a09e-46091a1e2995 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:23.450785 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc4bfc-1547-42a1-8ba9-78115f22fede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:23.451428 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc4bfc-1547-42a1-8ba9-78115f22fede None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:23.451701 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc4bfc-1547-42a1-8ba9-78115f22fede None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:23.451968 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc4bfc-1547-42a1-8ba9-78115f22fede None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:23.470893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30f90a-a11a-4e65-a09e-46091a1e2995 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:23.472370 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30f90a-a11a-4e65-a09e-46091a1e2995 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:23.473554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc4bfc-1547-42a1-8ba9-78115f22fede None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:23.475011 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc4bfc-1547-42a1-8ba9-78115f22fede None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:23.495177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-dc30f90a-a11a-4e65-a09e-46091a1e2995 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:23.495177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:23.495177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:23.495177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:23.495177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:23.495177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:23.495177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:23.495177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:23.495177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:23.495177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:23.495177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:23.495177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:23.495177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:23.495177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:23.495177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:23.496131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:23.497546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:23.497546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:23.497546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:23.497546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:23.497546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:23.497546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:23.497546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:23.497546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:23.497546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 344/692] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:23.502890 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30f90a-a11a-4e65-a09e-46091a1e2995 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-c6cc4bfc-1547-42a1-8ba9-78115f22fede None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:23.511166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:23.512084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:23.513221 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:23.513221 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:23.513221 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:23.517378 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 349/693] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:23.540353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc4bfc-1547-42a1-8ba9-78115f22fede None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:23.640631 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:23.665660 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30f90a-a11a-4e65-a09e-46091a1e2995 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:23.703280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47af84a-5420-48dc-8e31-e0d379d297f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:23.704313 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47af84a-5420-48dc-8e31-e0d379d297f6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:23.704787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47af84a-5420-48dc-8e31-e0d379d297f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:23.705248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47af84a-5420-48dc-8e31-e0d379d297f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:23.706156 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47af84a-5420-48dc-8e31-e0d379d297f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:23.706826 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47af84a-5420-48dc-8e31-e0d379d297f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:23.708555 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47af84a-5420-48dc-8e31-e0d379d297f6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:23.722539 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 345/694] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:23.733531 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47af84a-5420-48dc-8e31-e0d379d297f6 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:23.746335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc4bfc-1547-42a1-8ba9-78115f22fede None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:23.786923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:23.809777 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88ae37-53f4-409c-a6bf-89d4d68f9a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:23.811065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88ae37-53f4-409c-a6bf-89d4d68f9a98 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:23.811355 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88ae37-53f4-409c-a6bf-89d4d68f9a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:23.811639 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88ae37-53f4-409c-a6bf-89d4d68f9a98 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:23.812300 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88ae37-53f4-409c-a6bf-89d4d68f9a98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:23.812612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88ae37-53f4-409c-a6bf-89d4d68f9a98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:23.813992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88ae37-53f4-409c-a6bf-89d4d68f9a98 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:23.827510 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 350/695] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:23.834171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88ae37-53f4-409c-a6bf-89d4d68f9a98 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:23.874784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47af84a-5420-48dc-8e31-e0d379d297f6 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:23.948199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc218a21-1ea2-4fb8-b866-dac74093caf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:23.948856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc218a21-1ea2-4fb8-b866-dac74093caf3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:23.949066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc218a21-1ea2-4fb8-b866-dac74093caf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:23.949334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc218a21-1ea2-4fb8-b866-dac74093caf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:23.950869 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc218a21-1ea2-4fb8-b866-dac74093caf3 None admin] RBAC: Adding query filter params (role.id=3209a3d044b84a0ebd7cf1edf2da7120, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=a73720a950944fa08bf1276ea78423b4) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:23.951227 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc218a21-1ea2-4fb8-b866-dac74093caf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:23.952977 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc218a21-1ea2-4fb8-b866-dac74093caf3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:23.966856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 346/696] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:23 2022] GET /identity/v3/role_assignments?user.id=a73720a950944fa08bf1276ea78423b4&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=3209a3d044b84a0ebd7cf1edf2da7120 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:23.968341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af03d4-de7a-4e3d-b595-d357cf83bd90 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:23.968727 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af03d4-de7a-4e3d-b595-d357cf83bd90 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:23.969109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af03d4-de7a-4e3d-b595-d357cf83bd90 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:23.974775 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 347/697] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:23.976691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59375537-1122-4870-936b-fdffaaadd659 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:23.977045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59375537-1122-4870-936b-fdffaaadd659 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:23.977390 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59375537-1122-4870-936b-fdffaaadd659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:23.985365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:23.999867 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-59375537-1122-4870-936b-fdffaaadd659 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:24.001214 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-59375537-1122-4870-936b-fdffaaadd659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:24.029313 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88ae37-53f4-409c-a6bf-89d4d68f9a98 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:24.078435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 348/698] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:00:24.088625 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa916d9-3135-403f-8117-d58f38ac006c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:24.089527 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa916d9-3135-403f-8117-d58f38ac006c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:24.089943 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa916d9-3135-403f-8117-d58f38ac006c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:24.090994 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa916d9-3135-403f-8117-d58f38ac006c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:24.102166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa916d9-3135-403f-8117-d58f38ac006c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:24.103837 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa916d9-3135-403f-8117-d58f38ac006c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-eaa916d9-3135-403f-8117-d58f38ac006c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:24.128819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:24.130009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:24.131162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:24.131162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:24.131162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:24.131162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:24.131162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:24.131162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:24.131162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:24.131162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:24.131162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:00:24.131162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:24.133477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 351/699] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 23 06:00:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:24.139691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59375537-1122-4870-936b-fdffaaadd659 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:24.266313 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59375537-1122-4870-936b-fdffaaadd659 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:24.299286 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:24.304092 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f119334-90f7-481f-8464-3d230d0d511a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:24.305018 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f119334-90f7-481f-8464-3d230d0d511a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:24.305301 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f119334-90f7-481f-8464-3d230d0d511a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:24.305577 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f119334-90f7-481f-8464-3d230d0d511a None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:24.306324 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f119334-90f7-481f-8464-3d230d0d511a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:24.307583 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f119334-90f7-481f-8464-3d230d0d511a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:24.309034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f119334-90f7-481f-8464-3d230d0d511a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:24.329712 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 349/700] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 23 06:00:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:24.334846 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa916d9-3135-403f-8117-d58f38ac006c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:24.385159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:24.406549 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f00e2-4698-4061-bcc7-534877d76546 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:24.407527 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f00e2-4698-4061-bcc7-534877d76546 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:24.408292 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f00e2-4698-4061-bcc7-534877d76546 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:24.410563 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 350/701] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:00:24.416044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f02839-7564-456c-85bb-ab7e8a79d283 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:24.416575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f02839-7564-456c-85bb-ab7e8a79d283 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:24.417401 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f02839-7564-456c-85bb-ab7e8a79d283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:24.442890 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-b0f02839-7564-456c-85bb-ab7e8a79d283 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:24.444079 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b0f02839-7564-456c-85bb-ab7e8a79d283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:24.467907 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa916d9-3135-403f-8117-d58f38ac006c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:24.527156 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422c830-0a5c-41b4-952a-572d47a4d2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:24.528367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422c830-0a5c-41b4-952a-572d47a4d2a3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:24.529321 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422c830-0a5c-41b4-952a-572d47a4d2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:24.530056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422c830-0a5c-41b4-952a-572d47a4d2a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:24.533795 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 351/702] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 23 06:00:24.538546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f02839-7564-456c-85bb-ab7e8a79d283 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:24.675581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422c830-0a5c-41b4-952a-572d47a4d2a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:24.677036 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422c830-0a5c-41b4-952a-572d47a4d2a3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:24.724870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f02839-7564-456c-85bb-ab7e8a79d283 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:24.737734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:24.758827 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:24.797391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd461bb8-251f-4165-8ba8-9c8c1216961e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:24.798004 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd461bb8-251f-4165-8ba8-9c8c1216961e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:24.798283 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd461bb8-251f-4165-8ba8-9c8c1216961e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:24.798585 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd461bb8-251f-4165-8ba8-9c8c1216961e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:24.816245 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 352/703] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 06:00:24.820582 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd461bb8-251f-4165-8ba8-9c8c1216961e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:24.822068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd461bb8-251f-4165-8ba8-9c8c1216961e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:24.823415 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422c830-0a5c-41b4-952a-572d47a4d2a3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-dd461bb8-251f-4165-8ba8-9c8c1216961e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:24.856967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:24.858038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:24.863364 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:24.863364 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:24.863364 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:24.865259 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 352/704] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:24.875754 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd461bb8-251f-4165-8ba8-9c8c1216961e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:24.958440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422c830-0a5c-41b4-952a-572d47a4d2a3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:25.000366 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d67081-caf2-40f0-a4a4-a6d0c3997662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:25.001349 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d67081-caf2-40f0-a4a4-a6d0c3997662 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:25.001854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d67081-caf2-40f0-a4a4-a6d0c3997662 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:25.002324 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d67081-caf2-40f0-a4a4-a6d0c3997662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:25.009747 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:25.014099 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd461bb8-251f-4165-8ba8-9c8c1216961e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:25.016466 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d67081-caf2-40f0-a4a4-a6d0c3997662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:25.018133 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d67081-caf2-40f0-a4a4-a6d0c3997662 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-24d67081-caf2-40f0-a4a4-a6d0c3997662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:25.035142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:25.036208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:25.037097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:25.037097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:25.037097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:25.039344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 353/705] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:25.040464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d67081-caf2-40f0-a4a4-a6d0c3997662 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:25.056009 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e40ca8-dd07-4a4a-8f3d-c376afbf49a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:25.056559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e40ca8-dd07-4a4a-8f3d-c376afbf49a2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:25.056837 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e40ca8-dd07-4a4a-8f3d-c376afbf49a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:25.057105 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e40ca8-dd07-4a4a-8f3d-c376afbf49a2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:25.057684 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e40ca8-dd07-4a4a-8f3d-c376afbf49a2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:25.059890 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e40ca8-dd07-4a4a-8f3d-c376afbf49a2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-59e40ca8-dd07-4a4a-8f3d-c376afbf49a2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:25.076140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:25.077252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:25.078088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:25.078088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:25.078088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:25.078088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:25.078088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:25.078088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:25.078088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:25.078088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:25.078088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:25.078088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:25.078088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:25.078088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:25.078088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 06:00:25.078088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:25.078088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 353/706] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Aug 23 06:00:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:25.078895 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e40ca8-dd07-4a4a-8f3d-c376afbf49a2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:25.142580 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:25.217732 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d67081-caf2-40f0-a4a4-a6d0c3997662 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:25.234198 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e40ca8-dd07-4a4a-8f3d-c376afbf49a2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:25.266196 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b9e1c-4ba6-451d-8035-943811bc0bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:25.267270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b9e1c-4ba6-451d-8035-943811bc0bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:25.267547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b9e1c-4ba6-451d-8035-943811bc0bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:25.267816 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b9e1c-4ba6-451d-8035-943811bc0bd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:25.268724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b9e1c-4ba6-451d-8035-943811bc0bd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:25.269035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b9e1c-4ba6-451d-8035-943811bc0bd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:25.270668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b9e1c-4ba6-451d-8035-943811bc0bd4 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:25.276358 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e33d91-613b-4ed3-93c4-00c1ae7ae803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:25.277280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e33d91-613b-4ed3-93c4-00c1ae7ae803 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:25.277759 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e33d91-613b-4ed3-93c4-00c1ae7ae803 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:25.278347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e33d91-613b-4ed3-93c4-00c1ae7ae803 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:25.279276 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e33d91-613b-4ed3-93c4-00c1ae7ae803 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:25.279825 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e33d91-613b-4ed3-93c4-00c1ae7ae803 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:25.281619 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e33d91-613b-4ed3-93c4-00c1ae7ae803 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:25.285563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 354/707] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:25.286287 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b9e1c-4ba6-451d-8035-943811bc0bd4 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:25.295170 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 354/708] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:25.297266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb08d6-45a4-446a-abc1-4d274fa93f54 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:25.297761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb08d6-45a4-446a-abc1-4d274fa93f54 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:25.298238 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb08d6-45a4-446a-abc1-4d274fa93f54 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:25.300668 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 355/709] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:25.301881 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb08d6-45a4-446a-abc1-4d274fa93f54 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:25.346191 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:25.407312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:25.407507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b9e1c-4ba6-451d-8035-943811bc0bd4 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:25.414715 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb08d6-45a4-446a-abc1-4d274fa93f54 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:25.446863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75872a-36e3-4818-8c03-b408b9b2f3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:25.447800 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75872a-36e3-4818-8c03-b408b9b2f3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:25.448279 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75872a-36e3-4818-8c03-b408b9b2f3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:25.448719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75872a-36e3-4818-8c03-b408b9b2f3b0 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:25.449511 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75872a-36e3-4818-8c03-b408b9b2f3b0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:25.450504 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75872a-36e3-4818-8c03-b408b9b2f3b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:25.450839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22eb8f8-32ed-4605-9349-d7ab9a639a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:25.451498 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22eb8f8-32ed-4605-9349-d7ab9a639a5f None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:25.451775 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22eb8f8-32ed-4605-9349-d7ab9a639a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:25.452077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22eb8f8-32ed-4605-9349-d7ab9a639a5f None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:25.452715 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22eb8f8-32ed-4605-9349-d7ab9a639a5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:25.453077 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75872a-36e3-4818-8c03-b408b9b2f3b0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:25.454144 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22eb8f8-32ed-4605-9349-d7ab9a639a5f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:25.458774 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 355/710] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 23 06:00:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:25.459765 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75872a-36e3-4818-8c03-b408b9b2f3b0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:25.477662 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 356/711] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 06:00:25 2022] POST /identity/v3/users => generated 254 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:00:25.478925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c79c1-05d3-45c5-8e4d-52c36198f7ad None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:25.479209 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c79c1-05d3-45c5-8e4d-52c36198f7ad None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:25.479482 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c79c1-05d3-45c5-8e4d-52c36198f7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:25.504024 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c41c79c1-05d3-45c5-8e4d-52c36198f7ad None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:25.505931 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c41c79c1-05d3-45c5-8e4d-52c36198f7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:25.604356 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75872a-36e3-4818-8c03-b408b9b2f3b0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:25.622921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 357/712] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:25.624369 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c79c1-05d3-45c5-8e4d-52c36198f7ad None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:25.625447 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:25.656257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8132a-3850-472b-bd62-f57d60ffbfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:25.656904 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8132a-3850-472b-bd62-f57d60ffbfa6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:25.657183 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8132a-3850-472b-bd62-f57d60ffbfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:25.657452 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8132a-3850-472b-bd62-f57d60ffbfa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:25.667161 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8132a-3850-472b-bd62-f57d60ffbfa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:25.669411 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8132a-3850-472b-bd62-f57d60ffbfa6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:25.683942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-00f8132a-3850-472b-bd62-f57d60ffbfa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:25.683942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:25.683942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:25.683942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:25.683942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:25.683942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:25.683942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:25.683942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:25.683942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:25.683942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:25.683942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:25.683942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:25.683942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:25.683942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:25.683942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:25.685764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:25.686750 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:25.686750 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:25.686750 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:25.686750 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:25.686750 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:25.686750 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:25.686750 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:25.686750 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:25.686750 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 356/713] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:25.691822 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8132a-3850-472b-bd62-f57d60ffbfa6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:25.741194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:25.763330 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c79c1-05d3-45c5-8e4d-52c36198f7ad None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:25.804564 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36517860-101b-416e-b344-146191004e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:25.805585 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36517860-101b-416e-b344-146191004e39 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:25.806100 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36517860-101b-416e-b344-146191004e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:25.806604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36517860-101b-416e-b344-146191004e39 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:25.807427 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36517860-101b-416e-b344-146191004e39 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:25.807962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36517860-101b-416e-b344-146191004e39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:25.809947 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36517860-101b-416e-b344-146191004e39 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:25.816364 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 358/714] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 23 06:00:25 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:25.830023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8132a-3850-472b-bd62-f57d60ffbfa6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:25.876782 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125b9ac-e459-4eeb-a422-7e66b6a70608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:25.877699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125b9ac-e459-4eeb-a422-7e66b6a70608 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:25.878165 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125b9ac-e459-4eeb-a422-7e66b6a70608 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:25.878651 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125b9ac-e459-4eeb-a422-7e66b6a70608 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:25.879691 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125b9ac-e459-4eeb-a422-7e66b6a70608 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:25.880233 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125b9ac-e459-4eeb-a422-7e66b6a70608 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:25.881941 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125b9ac-e459-4eeb-a422-7e66b6a70608 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:25.897646 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 357/715] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:25.905202 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36517860-101b-416e-b344-146191004e39 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:25.992122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125b9ac-e459-4eeb-a422-7e66b6a70608 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:26.039013 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36517860-101b-416e-b344-146191004e39 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:26.048592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:26.086283 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34be57-9250-4c2c-810f-34e83aac9f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:26.087228 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34be57-9250-4c2c-810f-34e83aac9f41 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:26.088108 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34be57-9250-4c2c-810f-34e83aac9f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:26.088530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34be57-9250-4c2c-810f-34e83aac9f41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:26.089572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34be57-9250-4c2c-810f-34e83aac9f41 None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=91267a049343452eb208205ee3ea27c4) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:26.090171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34be57-9250-4c2c-810f-34e83aac9f41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:26.093384 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34be57-9250-4c2c-810f-34e83aac9f41 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:26.100910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 359/716] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:25 2022] GET /identity/v3/role_assignments?user.id=91267a049343452eb208205ee3ea27c4&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:26.169823 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125b9ac-e459-4eeb-a422-7e66b6a70608 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:26.227802 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643a5fb-1d52-4540-919e-117397097567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:26.228544 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643a5fb-1d52-4540-919e-117397097567 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:26.228889 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643a5fb-1d52-4540-919e-117397097567 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:26.229225 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643a5fb-1d52-4540-919e-117397097567 None admin] PATH_INFO: `/v3/services/network` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:26.229876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643a5fb-1d52-4540-919e-117397097567 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:26.231880 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643a5fb-1d52-4540-919e-117397097567 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-e643a5fb-1d52-4540-919e-117397097567 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:26.247650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:26.248971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:26.250025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:26.250025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:26.250025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:26.250025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:26.250025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:26.250025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:26.250025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:26.250025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:26.250025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:26.250025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:26.250025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:26.250025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:26.250025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 06:00:26.250025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:26.256882 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 358/717] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:25 2022] GET /identity/v3/services/network => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:26.263691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34be57-9250-4c2c-810f-34e83aac9f41 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:26.287121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:26.412872 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34be57-9250-4c2c-810f-34e83aac9f41 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:26.420538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd1762-9892-43fe-a084-27510720f709 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:26.420725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd1762-9892-43fe-a084-27510720f709 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:26.420997 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd1762-9892-43fe-a084-27510720f709 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:26.422471 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 359/718] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:26.428017 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774774b-6230-4065-93fd-3f29ec216309 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:26.428301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774774b-6230-4065-93fd-3f29ec216309 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:26.428575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774774b-6230-4065-93fd-3f29ec216309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:26.451434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-f774774b-6230-4065-93fd-3f29ec216309 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:26.452636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f774774b-6230-4065-93fd-3f29ec216309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:26.464832 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e38bb-b014-483f-b49e-76d8b041bbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:26.466030 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e38bb-b014-483f-b49e-76d8b041bbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:26.466302 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e38bb-b014-483f-b49e-76d8b041bbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:26.466598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e38bb-b014-483f-b49e-76d8b041bbf0 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:26.467251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e38bb-b014-483f-b49e-76d8b041bbf0 None admin] RBAC: Adding query filter params (name=network) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:26.467551 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e38bb-b014-483f-b49e-76d8b041bbf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:26.468960 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e38bb-b014-483f-b49e-76d8b041bbf0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:26.477654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 360/719] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 23 06:00:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:26.486001 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e38bb-b014-483f-b49e-76d8b041bbf0 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:26.582912 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:26.622434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 360/720] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:00:26.629502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e38bb-b014-483f-b49e-76d8b041bbf0 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:26.669800 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51596366-5b8d-419a-b84a-91cce48f7ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:26.670682 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51596366-5b8d-419a-b84a-91cce48f7ead None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:26.670919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51596366-5b8d-419a-b84a-91cce48f7ead None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:26.671201 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51596366-5b8d-419a-b84a-91cce48f7ead None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:26.671847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51596366-5b8d-419a-b84a-91cce48f7ead None admin] RBAC: Adding query filter params (type=network) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:26.672156 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51596366-5b8d-419a-b84a-91cce48f7ead None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:26.673686 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51596366-5b8d-419a-b84a-91cce48f7ead None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:26.678910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 361/721] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 23 06:00:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:27.017539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774774b-6230-4065-93fd-3f29ec216309 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:27.051505 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:27.130889 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774774b-6230-4065-93fd-3f29ec216309 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:27.174939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6f358-3c89-46da-9ef8-0d9238deeeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:27.175565 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6f358-3c89-46da-9ef8-0d9238deeeab None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:27.175842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6f358-3c89-46da-9ef8-0d9238deeeab None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:27.176110 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6f358-3c89-46da-9ef8-0d9238deeeab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:27.191665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6f358-3c89-46da-9ef8-0d9238deeeab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-aeb6f358-3c89-46da-9ef8-0d9238deeeab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:27.196332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:27.197242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:27.198067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:27.198067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:27.198067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:27.198067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:27.198067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:27.198067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:27.198067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:27.198067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:27.198067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:27.198067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:27.198067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:00:27.198067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:27.199208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 361/722] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:00:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:27.203675 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51596366-5b8d-419a-b84a-91cce48f7ead None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:27.332436 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51596366-5b8d-419a-b84a-91cce48f7ead None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:27.382293 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb23ea2-ab20-4012-8c7e-061d5a20205e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:27.382959 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb23ea2-ab20-4012-8c7e-061d5a20205e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:27.383241 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb23ea2-ab20-4012-8c7e-061d5a20205e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:27.383517 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb23ea2-ab20-4012-8c7e-061d5a20205e None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:27.384199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb23ea2-ab20-4012-8c7e-061d5a20205e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:27.384503 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb23ea2-ab20-4012-8c7e-061d5a20205e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:27.385887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb23ea2-ab20-4012-8c7e-061d5a20205e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:27.392185 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:27.402101 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 362/723] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:00:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:27.408421 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6f358-3c89-46da-9ef8-0d9238deeeab None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:27.540511 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:27.553672 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6f358-3c89-46da-9ef8-0d9238deeeab None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:27.609870 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f5dfe-f67c-41e8-904b-9de5044ada0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:27.610436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f5dfe-f67c-41e8-904b-9de5044ada0c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:27.610740 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f5dfe-f67c-41e8-904b-9de5044ada0c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:27.611014 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f5dfe-f67c-41e8-904b-9de5044ada0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:27.631283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f5dfe-f67c-41e8-904b-9de5044ada0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:27.632766 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f5dfe-f67c-41e8-904b-9de5044ada0c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-3d5f5dfe-f67c-41e8-904b-9de5044ada0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:27.649958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:27.650964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:27.651930 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:27.651930 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:27.651930 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:27.655491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 362/724] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:27.661161 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb23ea2-ab20-4012-8c7e-061d5a20205e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:27.768577 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb23ea2-ab20-4012-8c7e-061d5a20205e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:27.803328 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e34d4-6473-430f-b88c-d43b1450f930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:27.804308 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e34d4-6473-430f-b88c-d43b1450f930 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:27.804787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e34d4-6473-430f-b88c-d43b1450f930 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:27.805269 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e34d4-6473-430f-b88c-d43b1450f930 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:27.806102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e34d4-6473-430f-b88c-d43b1450f930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:27.806634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e34d4-6473-430f-b88c-d43b1450f930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:27.808215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e34d4-6473-430f-b88c-d43b1450f930 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:27.822321 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 363/725] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:27.831813 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f5dfe-f67c-41e8-904b-9de5044ada0c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:27.996232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f5dfe-f67c-41e8-904b-9de5044ada0c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:28.015508 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549dcdd-0041-4202-968f-5d7bd0836c76 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:28.015815 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549dcdd-0041-4202-968f-5d7bd0836c76 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:28.016055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549dcdd-0041-4202-968f-5d7bd0836c76 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:28.017574 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 364/726] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:00:28.022787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd17ba7-5122-43f6-87de-4d748019ea5d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:28.023066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd17ba7-5122-43f6-87de-4d748019ea5d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:28.023335 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd17ba7-5122-43f6-87de-4d748019ea5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:28.048110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5dd17ba7-5122-43f6-87de-4d748019ea5d None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:28.049351 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5dd17ba7-5122-43f6-87de-4d748019ea5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:28.054852 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63ce1a-d722-42ce-b8b4-b2e967778c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:28.055698 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63ce1a-d722-42ce-b8b4-b2e967778c71 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:28.056009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63ce1a-d722-42ce-b8b4-b2e967778c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:28.056468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63ce1a-d722-42ce-b8b4-b2e967778c71 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:28.072961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63ce1a-d722-42ce-b8b4-b2e967778c71 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:28.074405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63ce1a-d722-42ce-b8b4-b2e967778c71 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:28.080858 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-6f63ce1a-d722-42ce-b8b4-b2e967778c71 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:28.093838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:28.094876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:28.095809 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:28.095809 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:28.095809 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:28.095809 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:28.095809 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:28.095809 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:28.095809 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:28.095809 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:28.095809 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:00:28.095809 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:28.098273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 363/727] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:00:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:28.102758 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63ce1a-d722-42ce-b8b4-b2e967778c71 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:28.160608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:28.183259 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 365/728] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:28.261889 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63ce1a-d722-42ce-b8b4-b2e967778c71 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:28.301386 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542dbda3-6560-40d0-b902-807547410656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:28.302002 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542dbda3-6560-40d0-b902-807547410656 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:28.302274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542dbda3-6560-40d0-b902-807547410656 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:28.302555 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542dbda3-6560-40d0-b902-807547410656 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:28.303290 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542dbda3-6560-40d0-b902-807547410656 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:28.303682 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542dbda3-6560-40d0-b902-807547410656 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:28.305187 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542dbda3-6560-40d0-b902-807547410656 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:28.323930 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 364/729] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 23 06:00:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:28.331704 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd17ba7-5122-43f6-87de-4d748019ea5d None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:28.365889 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783adcc1-7a67-484b-a410-7d839279a82e None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:28.366210 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783adcc1-7a67-484b-a410-7d839279a82e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:28.366438 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783adcc1-7a67-484b-a410-7d839279a82e None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:28.368309 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 365/730] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:00:28.373316 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79afbc-1e56-49c4-8b9f-ba87d7fc7a10 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:28.373595 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79afbc-1e56-49c4-8b9f-ba87d7fc7a10 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:28.373863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79afbc-1e56-49c4-8b9f-ba87d7fc7a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:28.404390 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ad79afbc-1e56-49c4-8b9f-ba87d7fc7a10 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:28.405636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ad79afbc-1e56-49c4-8b9f-ba87d7fc7a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:28.486580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:28.517017 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 366/731] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 06:00:28.527319 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd17ba7-5122-43f6-87de-4d748019ea5d None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:28.580044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ddf96-f3a0-4c4c-bd97-39c4d92f90ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:28.580662 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ddf96-f3a0-4c4c-bd97-39c4d92f90ae None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:28.580948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ddf96-f3a0-4c4c-bd97-39c4d92f90ae None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:28.581221 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ddf96-f3a0-4c4c-bd97-39c4d92f90ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:28.593303 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79afbc-1e56-49c4-8b9f-ba87d7fc7a10 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:28.652231 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:28.666074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ddf96-f3a0-4c4c-bd97-39c4d92f90ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:28.667631 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ddf96-f3a0-4c4c-bd97-39c4d92f90ae None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:28.731497 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79afbc-1e56-49c4-8b9f-ba87d7fc7a10 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:28.769877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8128e-e656-45d2-a332-0f25f2b74d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:28.770439 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8128e-e656-45d2-a332-0f25f2b74d22 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:28.770754 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8128e-e656-45d2-a332-0f25f2b74d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:28.771030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8128e-e656-45d2-a332-0f25f2b74d22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:28.773017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 366/732] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 06:00:28.774286 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca76c4-a31c-4ef2-9278-23e0de1a0a7b None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:28.779890 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca76c4-a31c-4ef2-9278-23e0de1a0a7b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:28.780225 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca76c4-a31c-4ef2-9278-23e0de1a0a7b None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:28.781771 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 367/733] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:28.782377 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca76c4-a31c-4ef2-9278-23e0de1a0a7b None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:28.783431 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8128e-e656-45d2-a332-0f25f2b74d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-ecb8128e-e656-45d2-a332-0f25f2b74d22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:28.788512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:28.789512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:28.790335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:28.790335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:28.790335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:28.790335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:28.790335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:28.790335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:28.790335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:28.790335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:28.790335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:28.790335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:28.790335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:28.790335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:28.790335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 367/734] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:28.792153 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa7408-a928-4e56-a5be-8d6d90ee2270 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:28.794623 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa7408-a928-4e56-a5be-8d6d90ee2270 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:28.794623 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa7408-a928-4e56-a5be-8d6d90ee2270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:28.813170 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-5dfa7408-a928-4e56-a5be-8d6d90ee2270 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:28.814535 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5dfa7408-a928-4e56-a5be-8d6d90ee2270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:28.864300 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:28.882860 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:28.902541 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 368/735] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:28.902541 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa7408-a928-4e56-a5be-8d6d90ee2270 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:28.908526 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca76c4-a31c-4ef2-9278-23e0de1a0a7b None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:28.946475 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd3231-c9e2-47a0-99ee-d504cb85edbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:28.947193 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd3231-c9e2-47a0-99ee-d504cb85edbc None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:28.950473 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd3231-c9e2-47a0-99ee-d504cb85edbc None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:28.950473 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd3231-c9e2-47a0-99ee-d504cb85edbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:28.970124 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd3231-c9e2-47a0-99ee-d504cb85edbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:28.972456 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd3231-c9e2-47a0-99ee-d504cb85edbc None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-08fd3231-c9e2-47a0-99ee-d504cb85edbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:28.993378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:28.994413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:28.995367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:28.995367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:28.995367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:28.998902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 368/736] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:29.000023 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd3231-c9e2-47a0-99ee-d504cb85edbc None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:29.021542 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa7408-a928-4e56-a5be-8d6d90ee2270 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:29.061832 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271284cd-80fa-4d4d-b1cc-f20b2065ccf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:29.063279 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271284cd-80fa-4d4d-b1cc-f20b2065ccf5 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:29.063796 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271284cd-80fa-4d4d-b1cc-f20b2065ccf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:29.064320 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271284cd-80fa-4d4d-b1cc-f20b2065ccf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:29.065319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271284cd-80fa-4d4d-b1cc-f20b2065ccf5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:29.065824 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271284cd-80fa-4d4d-b1cc-f20b2065ccf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:29.067551 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271284cd-80fa-4d4d-b1cc-f20b2065ccf5 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:29.078115 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 369/737] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:29.079409 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271284cd-80fa-4d4d-b1cc-f20b2065ccf5 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:29.120143 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd3231-c9e2-47a0-99ee-d504cb85edbc None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:29.128292 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:29.157659 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef47f69-3943-4f49-9b0a-aabaead2b9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:29.158270 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef47f69-3943-4f49-9b0a-aabaead2b9b7 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:29.158586 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef47f69-3943-4f49-9b0a-aabaead2b9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:29.158861 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef47f69-3943-4f49-9b0a-aabaead2b9b7 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:29.159502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef47f69-3943-4f49-9b0a-aabaead2b9b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:29.160882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef47f69-3943-4f49-9b0a-aabaead2b9b7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:29.186545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 369/738] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 23 06:00:28 2022] POST /identity/v3/services => generated 251 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:00:29.186545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef47f69-3943-4f49-9b0a-aabaead2b9b7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:29.187327 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:29.216690 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271284cd-80fa-4d4d-b1cc-f20b2065ccf5 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:29.275338 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17906ee9-5313-4548-ab3b-251b3fb76d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:29.276371 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17906ee9-5313-4548-ab3b-251b3fb76d7a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:29.276863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17906ee9-5313-4548-ab3b-251b3fb76d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:29.277336 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17906ee9-5313-4548-ab3b-251b3fb76d7a None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:29.278869 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17906ee9-5313-4548-ab3b-251b3fb76d7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:29.279688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17906ee9-5313-4548-ab3b-251b3fb76d7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:29.281379 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17906ee9-5313-4548-ab3b-251b3fb76d7a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:29.295529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 370/739] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:29.296765 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17906ee9-5313-4548-ab3b-251b3fb76d7a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:29.315256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef47f69-3943-4f49-9b0a-aabaead2b9b7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:29.357526 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c016738-671b-4523-9941-e34bbd535f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:29.358344 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c016738-671b-4523-9941-e34bbd535f31 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:29.358506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c016738-671b-4523-9941-e34bbd535f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:29.359435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c016738-671b-4523-9941-e34bbd535f31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:29.373095 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c016738-671b-4523-9941-e34bbd535f31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:29.374678 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c016738-671b-4523-9941-e34bbd535f31 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-5c016738-671b-4523-9941-e34bbd535f31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:29.392038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:29.394226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:29.395837 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:29.395837 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:29.395837 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:29.395837 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 370/740] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:29.395837 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c016738-671b-4523-9941-e34bbd535f31 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:29.440532 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:29.447361 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17906ee9-5313-4548-ab3b-251b3fb76d7a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:29.464975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:29.487382 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd785749-4372-41ea-9200-f40a6ed36cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:29.488293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd785749-4372-41ea-9200-f40a6ed36cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:29.488605 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd785749-4372-41ea-9200-f40a6ed36cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:29.488834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd785749-4372-41ea-9200-f40a6ed36cd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:29.500066 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd785749-4372-41ea-9200-f40a6ed36cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-dd785749-4372-41ea-9200-f40a6ed36cd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:29.505261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:29.506384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:29.507529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:29.507529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:29.507529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:29.507529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:29.507529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:29.507529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:29.507529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:29.507529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:29.507529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:29.507529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:29.507529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:00:29.507529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:29.507529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 371/741] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:00:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:29.508233 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b1e2e-c12e-4e86-abd3-cbe303d15653 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:29.508233 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b1e2e-c12e-4e86-abd3-cbe303d15653 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:29.508508 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b1e2e-c12e-4e86-abd3-cbe303d15653 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:29.509917 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 372/742] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:29.510445 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b1e2e-c12e-4e86-abd3-cbe303d15653 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:29.524154 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c016738-671b-4523-9941-e34bbd535f31 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:29.566015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03e68d-471a-4a4b-890a-cb6d68503e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:29.566865 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03e68d-471a-4a4b-890a-cb6d68503e3b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:29.567151 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03e68d-471a-4a4b-890a-cb6d68503e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:29.567427 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03e68d-471a-4a4b-890a-cb6d68503e3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:29.578158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03e68d-471a-4a4b-890a-cb6d68503e3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:29.579922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03e68d-471a-4a4b-890a-cb6d68503e3b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:29.592252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-4e03e68d-471a-4a4b-890a-cb6d68503e3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:29.592252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:29.592252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:29.592252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:29.592252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:29.592252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:29.592252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:29.592252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:29.592252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:29.592252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:29.592252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:29.592252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:29.592252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:29.592252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:29.592252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:29.593199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:29.594112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:29.594112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:29.594112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:29.594112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:29.594112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:29.594112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:29.594112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:29.594112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:29.594112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 371/743] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:29.595362 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bbd8b-49fa-4fe8-b889-3217a6ff8663 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:29.595362 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bbd8b-49fa-4fe8-b889-3217a6ff8663 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:29.595362 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bbd8b-49fa-4fe8-b889-3217a6ff8663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:29.614002 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1a0bbd8b-49fa-4fe8-b889-3217a6ff8663 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:29.615206 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1a0bbd8b-49fa-4fe8-b889-3217a6ff8663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:29.635340 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b1e2e-c12e-4e86-abd3-cbe303d15653 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:29.674646 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708dfb8-589f-421d-8c0c-b2f6e770a446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:29.675531 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708dfb8-589f-421d-8c0c-b2f6e770a446 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:29.675955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708dfb8-589f-421d-8c0c-b2f6e770a446 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:29.676371 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708dfb8-589f-421d-8c0c-b2f6e770a446 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:29.677149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708dfb8-589f-421d-8c0c-b2f6e770a446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:29.677594 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708dfb8-589f-421d-8c0c-b2f6e770a446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:29.679144 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708dfb8-589f-421d-8c0c-b2f6e770a446 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:29.691167 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 372/744] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 06:00:29.692155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bbd8b-49fa-4fe8-b889-3217a6ff8663 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:29.697387 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 373/745] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:29.697387 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708dfb8-589f-421d-8c0c-b2f6e770a446 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:29.730307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:29.752060 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:29.823509 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bbd8b-49fa-4fe8-b889-3217a6ff8663 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:29.833592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708dfb8-589f-421d-8c0c-b2f6e770a446 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:29.861150 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffbe03-cddd-4526-ad67-079babdbb00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:29.862146 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffbe03-cddd-4526-ad67-079babdbb00b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:29.862671 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffbe03-cddd-4526-ad67-079babdbb00b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:29.863122 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffbe03-cddd-4526-ad67-079babdbb00b None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:29.863961 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffbe03-cddd-4526-ad67-079babdbb00b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:29.864480 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffbe03-cddd-4526-ad67-079babdbb00b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:29.866133 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffbe03-cddd-4526-ad67-079babdbb00b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:29.870190 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1bff0-cd87-4c39-8347-3440902c3e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:29.870967 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1bff0-cd87-4c39-8347-3440902c3e69 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:29.871305 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1bff0-cd87-4c39-8347-3440902c3e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:29.871694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1bff0-cd87-4c39-8347-3440902c3e69 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:29.872439 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1bff0-cd87-4c39-8347-3440902c3e69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:29.872803 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1bff0-cd87-4c39-8347-3440902c3e69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:29.874284 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1bff0-cd87-4c39-8347-3440902c3e69 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:29.877541 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 373/746] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:00:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:29.878598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffbe03-cddd-4526-ad67-079babdbb00b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:29.889193 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 374/747] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:29.889776 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1bff0-cd87-4c39-8347-3440902c3e69 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:29.970205 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:29.994963 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1bff0-cd87-4c39-8347-3440902c3e69 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:29.996739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffbe03-cddd-4526-ad67-079babdbb00b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:30.004950 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:30.031877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108db8b-17d5-4de3-8ffe-204ff592ef1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:30.032497 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108db8b-17d5-4de3-8ffe-204ff592ef1b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:30.032778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108db8b-17d5-4de3-8ffe-204ff592ef1b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:30.033029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108db8b-17d5-4de3-8ffe-204ff592ef1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:30.037118 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03b4af-fef9-46e0-b8aa-ac508976de36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:30.037752 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03b4af-fef9-46e0-b8aa-ac508976de36 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:30.038031 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03b4af-fef9-46e0-b8aa-ac508976de36 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:30.038303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03b4af-fef9-46e0-b8aa-ac508976de36 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:30.043551 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108db8b-17d5-4de3-8ffe-204ff592ef1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:30.045247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108db8b-17d5-4de3-8ffe-204ff592ef1b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:30.047150 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03b4af-fef9-46e0-b8aa-ac508976de36 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:30.048599 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03b4af-fef9-46e0-b8aa-ac508976de36 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-e108db8b-17d5-4de3-8ffe-204ff592ef1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:30.059911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:30.061023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:30.061898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:30.061898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:30.061898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-0f03b4af-fef9-46e0-b8aa-ac508976de36 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:30.062497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:30.063365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:30.064278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:30.064278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:30.064278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:30.064278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:30.064278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:30.064278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:30.064278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:30.064278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:30.064278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 06:00:30.064278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:30.064827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 375/748] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:30.064827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108db8b-17d5-4de3-8ffe-204ff592ef1b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:30.070978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 374/749] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 23 06:00:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:30.070978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03b4af-fef9-46e0-b8aa-ac508976de36 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:30.172804 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108db8b-17d5-4de3-8ffe-204ff592ef1b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:30.176144 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03b4af-fef9-46e0-b8aa-ac508976de36 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:30.201970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:30.210466 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ca780-26a3-468d-b298-5c30f912f97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:30.211374 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ca780-26a3-468d-b298-5c30f912f97c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:30.211852 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ca780-26a3-468d-b298-5c30f912f97c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:30.212318 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ca780-26a3-468d-b298-5c30f912f97c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:30.212748 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5005a9d-dc68-47b6-ac0d-ce807972fce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:30.213350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5005a9d-dc68-47b6-ac0d-ce807972fce3 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:30.213628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5005a9d-dc68-47b6-ac0d-ce807972fce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:30.213899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5005a9d-dc68-47b6-ac0d-ce807972fce3 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:30.214166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ca780-26a3-468d-b298-5c30f912f97c None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=a73720a950944fa08bf1276ea78423b4) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:30.214553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5005a9d-dc68-47b6-ac0d-ce807972fce3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:30.214922 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ca780-26a3-468d-b298-5c30f912f97c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:30.216105 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5005a9d-dc68-47b6-ac0d-ce807972fce3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:30.216806 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ca780-26a3-468d-b298-5c30f912f97c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:30.223237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 376/750] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:30 2022] GET /identity/v3/role_assignments?user.id=a73720a950944fa08bf1276ea78423b4&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:30.224483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ca780-26a3-468d-b298-5c30f912f97c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:30.249661 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 375/751] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 23 06:00:30 2022] POST /identity/v3/services => generated 240 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:00:30.250790 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5005a9d-dc68-47b6-ac0d-ce807972fce3 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:30.310333 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:30.442119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ca780-26a3-468d-b298-5c30f912f97c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:30.461498 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5005a9d-dc68-47b6-ac0d-ce807972fce3 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:30.484950 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfd332-970a-49e8-88bb-232b4ae2dcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:30.485566 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfd332-970a-49e8-88bb-232b4ae2dcf2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:30.485844 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfd332-970a-49e8-88bb-232b4ae2dcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:30.486124 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfd332-970a-49e8-88bb-232b4ae2dcf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:30.488105 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfd332-970a-49e8-88bb-232b4ae2dcf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:30.488105 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfd332-970a-49e8-88bb-232b4ae2dcf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:30.489493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfd332-970a-49e8-88bb-232b4ae2dcf2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:30.505186 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 377/752] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:30.509948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17ea64-ae0c-4e93-ab5d-6adfa1412ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:30.509948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17ea64-ae0c-4e93-ab5d-6adfa1412ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:30.509948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17ea64-ae0c-4e93-ab5d-6adfa1412ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:30.509948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17ea64-ae0c-4e93-ab5d-6adfa1412ae0 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:30.510737 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfd332-970a-49e8-88bb-232b4ae2dcf2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:30.511365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17ea64-ae0c-4e93-ab5d-6adfa1412ae0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:30.511682 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17ea64-ae0c-4e93-ab5d-6adfa1412ae0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:30.513170 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17ea64-ae0c-4e93-ab5d-6adfa1412ae0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:30.532926 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 376/753] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 23 06:00:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:30.538340 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17ea64-ae0c-4e93-ab5d-6adfa1412ae0 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:30.577632 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:30.631354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfd332-970a-49e8-88bb-232b4ae2dcf2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:30.651639 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:30.673004 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17ea64-ae0c-4e93-ab5d-6adfa1412ae0 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:30.687763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c6de2-86c5-4222-b4a9-5b5deec8122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:30.688427 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c6de2-86c5-4222-b4a9-5b5deec8122e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:30.688709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c6de2-86c5-4222-b4a9-5b5deec8122e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:30.688962 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c6de2-86c5-4222-b4a9-5b5deec8122e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:30.704479 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c6de2-86c5-4222-b4a9-5b5deec8122e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:30.709022 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c6de2-86c5-4222-b4a9-5b5deec8122e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-d97c6de2-86c5-4222-b4a9-5b5deec8122e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:30.726720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:30.727708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:30.728654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:30.728654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:30.728654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:30.728654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:30.728654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:30.728654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:30.728654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:30.728654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:30.728654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:00:30.728654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:30.738196 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 378/754] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 23 06:00:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:30.744239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302960f2-fe96-407d-9deb-f68c2db021df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:30.745342 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302960f2-fe96-407d-9deb-f68c2db021df None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:30.745827 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302960f2-fe96-407d-9deb-f68c2db021df None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:30.746301 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302960f2-fe96-407d-9deb-f68c2db021df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:30.748802 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c6de2-86c5-4222-b4a9-5b5deec8122e None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:30.877942 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302960f2-fe96-407d-9deb-f68c2db021df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:30.879847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302960f2-fe96-407d-9deb-f68c2db021df None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:30.933589 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c6de2-86c5-4222-b4a9-5b5deec8122e None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:30.953565 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:30.990197 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d6e3a-5000-4d17-8ab5-dfc432c9cf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:30.990851 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d6e3a-5000-4d17-8ab5-dfc432c9cf47 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:30.991127 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d6e3a-5000-4d17-8ab5-dfc432c9cf47 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:30.991395 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d6e3a-5000-4d17-8ab5-dfc432c9cf47 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:30.992130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d6e3a-5000-4d17-8ab5-dfc432c9cf47 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:30.992430 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d6e3a-5000-4d17-8ab5-dfc432c9cf47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:30.994133 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d6e3a-5000-4d17-8ab5-dfc432c9cf47 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:31.032094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 379/755] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 23 06:00:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:31.040470 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d6e3a-5000-4d17-8ab5-dfc432c9cf47 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:31.044325 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:31.065364 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 377/756] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:00:31.083587 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302960f2-fe96-407d-9deb-f68c2db021df None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:31.169382 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d6e3a-5000-4d17-8ab5-dfc432c9cf47 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:31.238739 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53390d-450f-4eac-bf4d-1a00897dff75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:31.239298 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53390d-450f-4eac-bf4d-1a00897dff75 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:31.239573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53390d-450f-4eac-bf4d-1a00897dff75 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:31.239843 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53390d-450f-4eac-bf4d-1a00897dff75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:31.252822 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53390d-450f-4eac-bf4d-1a00897dff75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:31.254481 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53390d-450f-4eac-bf4d-1a00897dff75 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:31.261295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302960f2-fe96-407d-9deb-f68c2db021df None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-fa53390d-450f-4eac-bf4d-1a00897dff75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:31.270925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:31.271902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:31.272776 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:31.272776 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:31.272776 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:31.272776 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 380/757] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:31.281029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53390d-450f-4eac-bf4d-1a00897dff75 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:31.315036 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:31.329657 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af988f5-b9e6-416e-b0d0-da16ed341eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:31.330666 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af988f5-b9e6-416e-b0d0-da16ed341eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:31.331184 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af988f5-b9e6-416e-b0d0-da16ed341eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:31.331934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af988f5-b9e6-416e-b0d0-da16ed341eb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:31.343193 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af988f5-b9e6-416e-b0d0-da16ed341eb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:31.345024 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af988f5-b9e6-416e-b0d0-da16ed341eb1 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-2af988f5-b9e6-416e-b0d0-da16ed341eb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:31.360870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:31.361846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:31.362776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:31.362776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:31.362776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:31.364825 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 378/758] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:31.366236 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e81521-0a3d-4535-ae84-9e17f9168af5 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:31.366750 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e81521-0a3d-4535-ae84-9e17f9168af5 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:31.367238 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e81521-0a3d-4535-ae84-9e17f9168af5 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:31.369257 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 379/759] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:31.370906 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e81521-0a3d-4535-ae84-9e17f9168af5 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:31.403985 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53390d-450f-4eac-bf4d-1a00897dff75 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:31.407454 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:31.443169 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e625ed2-8b9d-44d2-b044-1fa9587c17a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:31.443780 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e625ed2-8b9d-44d2-b044-1fa9587c17a6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:31.444058 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e625ed2-8b9d-44d2-b044-1fa9587c17a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:31.444329 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e625ed2-8b9d-44d2-b044-1fa9587c17a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:31.445023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e625ed2-8b9d-44d2-b044-1fa9587c17a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:31.445316 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e625ed2-8b9d-44d2-b044-1fa9587c17a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:31.446768 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e625ed2-8b9d-44d2-b044-1fa9587c17a6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:31.464940 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 381/760] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:31.464940 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ae477-bf4d-4ad0-bf4e-a43a43200e9f None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:31.464940 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ae477-bf4d-4ad0-bf4e-a43a43200e9f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:31.464940 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ae477-bf4d-4ad0-bf4e-a43a43200e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:31.482487 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-960ae477-bf4d-4ad0-bf4e-a43a43200e9f None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:31.483920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-960ae477-bf4d-4ad0-bf4e-a43a43200e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:31.494265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e81521-0a3d-4535-ae84-9e17f9168af5 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:31.531368 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830faaa-5881-46dc-ae19-1d85ccfc816a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:31.532304 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830faaa-5881-46dc-ae19-1d85ccfc816a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:31.532820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830faaa-5881-46dc-ae19-1d85ccfc816a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:31.533293 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830faaa-5881-46dc-ae19-1d85ccfc816a None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:31.534137 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830faaa-5881-46dc-ae19-1d85ccfc816a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:31.534675 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830faaa-5881-46dc-ae19-1d85ccfc816a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:31.536282 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830faaa-5881-46dc-ae19-1d85ccfc816a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:31.551759 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 380/761] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:31.553015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830faaa-5881-46dc-ae19-1d85ccfc816a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:31.564171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 382/762] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:31.565234 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ae477-bf4d-4ad0-bf4e-a43a43200e9f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:31.621910 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:31.657281 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:31.669844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830faaa-5881-46dc-ae19-1d85ccfc816a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:31.687869 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ae477-bf4d-4ad0-bf4e-a43a43200e9f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:31.734539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896915ac-aac3-45e6-ac69-6044482b40fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:31.734539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896915ac-aac3-45e6-ac69-6044482b40fc None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:31.734539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896915ac-aac3-45e6-ac69-6044482b40fc None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:31.734539 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896915ac-aac3-45e6-ac69-6044482b40fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:31.748127 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896915ac-aac3-45e6-ac69-6044482b40fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:31.751546 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896915ac-aac3-45e6-ac69-6044482b40fc None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:31.766157 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a3b2c-7974-4b7c-815f-ee83d81c1dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:31.766834 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a3b2c-7974-4b7c-815f-ee83d81c1dfa None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:31.767123 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a3b2c-7974-4b7c-815f-ee83d81c1dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:31.767393 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a3b2c-7974-4b7c-815f-ee83d81c1dfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:31.776493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-896915ac-aac3-45e6-ac69-6044482b40fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:31.776493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:31.776493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:31.776493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:31.776493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:31.776493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:31.776493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:31.776493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:31.776493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:31.776493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:31.776493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:31.776493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:31.776493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:31.776493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:31.776493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:31.777419 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:31.779433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:31.779433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:31.779433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:31.779433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:31.779433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:31.779433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:31.779433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:31.779433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:31.783225 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a3b2c-7974-4b7c-815f-ee83d81c1dfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:31.784682 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a3b2c-7974-4b7c-815f-ee83d81c1dfa None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:31.785381 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 383/763] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:31.787963 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896915ac-aac3-45e6-ac69-6044482b40fc None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:31.810998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-848a3b2c-7974-4b7c-815f-ee83d81c1dfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:31.810998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:31.810998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:31.810998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:31.810998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:31.810998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:31.810998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:31.810998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:31.810998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:31.810998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:31.810998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:31.810998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:31.810998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:31.810998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:31.810998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:31.813068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:31.814255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:31.814255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:31.814255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:31.814255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:31.814255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:31.814255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:31.814255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:31.814255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:31.814255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 381/764] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:31.820325 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a3b2c-7974-4b7c-815f-ee83d81c1dfa None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:31.953381 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a3b2c-7974-4b7c-815f-ee83d81c1dfa None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:31.960676 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:31.993583 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896915ac-aac3-45e6-ac69-6044482b40fc None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:31.998557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41fa22-b578-4899-8f39-676023059862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:31.999132 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41fa22-b578-4899-8f39-676023059862 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:31.999409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41fa22-b578-4899-8f39-676023059862 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:31.999683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41fa22-b578-4899-8f39-676023059862 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:32.000385 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41fa22-b578-4899-8f39-676023059862 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:32.000687 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41fa22-b578-4899-8f39-676023059862 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:32.002421 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41fa22-b578-4899-8f39-676023059862 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:32.018849 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 382/765] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:32.019714 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41fa22-b578-4899-8f39-676023059862 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:32.029100 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b64cf-8154-4b59-9672-53e84b04c4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:32.029705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b64cf-8154-4b59-9672-53e84b04c4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:32.029978 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b64cf-8154-4b59-9672-53e84b04c4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:32.030243 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b64cf-8154-4b59-9672-53e84b04c4e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:32.030973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b64cf-8154-4b59-9672-53e84b04c4e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:32.031268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b64cf-8154-4b59-9672-53e84b04c4e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:32.032679 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b64cf-8154-4b59-9672-53e84b04c4e9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:32.035745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:32.045251 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 384/766] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:32.046038 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b64cf-8154-4b59-9672-53e84b04c4e9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:32.139274 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41fa22-b578-4899-8f39-676023059862 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:32.147824 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b64cf-8154-4b59-9672-53e84b04c4e9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:32.177536 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:32.180147 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7852c-b403-428f-8c55-c84e8ac5e67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:32.181952 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7852c-b403-428f-8c55-c84e8ac5e67b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:32.182406 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7852c-b403-428f-8c55-c84e8ac5e67b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:32.182754 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32cdd0-1c5a-40fc-8159-d52e4f9822f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:32.183061 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7852c-b403-428f-8c55-c84e8ac5e67b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:32.183402 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32cdd0-1c5a-40fc-8159-d52e4f9822f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:32.183676 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32cdd0-1c5a-40fc-8159-d52e4f9822f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:32.184658 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32cdd0-1c5a-40fc-8159-d52e4f9822f5 None admin] PATH_INFO: `/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/91267a049343452eb208205ee3ea27c4/roles/f6acb6fdca5a44de9ba00b2b0d4d3202` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:32.184831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7852c-b403-428f-8c55-c84e8ac5e67b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:32.186430 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7852c-b403-428f-8c55-c84e8ac5e67b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:32.186719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32cdd0-1c5a-40fc-8159-d52e4f9822f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0b43dead1664475b9dd52113e08d7ed, user_id=91267a049343452eb208205ee3ea27c4, role_id=f6acb6fdca5a44de9ba00b2b0d4d3202)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:32.188131 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32cdd0-1c5a-40fc-8159-d52e4f9822f5 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-8bb7852c-b403-428f-8c55-c84e8ac5e67b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:32.197341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:32.198717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:32.199838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:32.199838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:32.199838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:32.199838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:32.199838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:32.199838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:32.199838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:32.199838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:32.199838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:32.199838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:32.199838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:32.199838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:32.199838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 06:00:32.199838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:32.202807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 383/767] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Aug 23 06:00:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:32.203854 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7852c-b403-428f-8c55-c84e8ac5e67b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:32.208683 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 385/768] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:00:32 2022] PUT /identity/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/91267a049343452eb208205ee3ea27c4/roles/f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:00:32.209892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32cdd0-1c5a-40fc-8159-d52e4f9822f5 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:32.298721 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:32.323133 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32cdd0-1c5a-40fc-8159-d52e4f9822f5 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:32.328344 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7852c-b403-428f-8c55-c84e8ac5e67b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:32.360073 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0f1bd-fa0e-4108-b6b9-fcfe8182bcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:32.360738 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0f1bd-fa0e-4108-b6b9-fcfe8182bcae None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:32.361159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0f1bd-fa0e-4108-b6b9-fcfe8182bcae None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:32.361329 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0f1bd-fa0e-4108-b6b9-fcfe8182bcae None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:32.361963 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0f1bd-fa0e-4108-b6b9-fcfe8182bcae None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:32.362264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0f1bd-fa0e-4108-b6b9-fcfe8182bcae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:32.364195 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0f1bd-fa0e-4108-b6b9-fcfe8182bcae None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:32.369518 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 386/769] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 23 06:00:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:32.369894 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea42039-b280-487f-9c9e-b7daa59d9970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:32.370716 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea42039-b280-487f-9c9e-b7daa59d9970 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:32.371136 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea42039-b280-487f-9c9e-b7daa59d9970 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:32.372133 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea42039-b280-487f-9c9e-b7daa59d9970 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:32.373128 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea42039-b280-487f-9c9e-b7daa59d9970 None admin] RBAC: Adding query filter params (role.id=3209a3d044b84a0ebd7cf1edf2da7120, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=078a012fea464954b0632a6ce86d7cec) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:32.373588 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea42039-b280-487f-9c9e-b7daa59d9970 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:32.375559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea42039-b280-487f-9c9e-b7daa59d9970 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:32.376086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0f1bd-fa0e-4108-b6b9-fcfe8182bcae None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:32.387002 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 384/770] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:32 2022] GET /identity/v3/role_assignments?user.id=078a012fea464954b0632a6ce86d7cec&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=3209a3d044b84a0ebd7cf1edf2da7120 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:32.442785 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:32.502781 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0f1bd-fa0e-4108-b6b9-fcfe8182bcae None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:32.545813 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c49c2-90aa-46c6-817b-536787e69ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:32.546868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c49c2-90aa-46c6-817b-536787e69ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:32.547292 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c49c2-90aa-46c6-817b-536787e69ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:32.547710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c49c2-90aa-46c6-817b-536787e69ad9 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:32.548491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c49c2-90aa-46c6-817b-536787e69ad9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:32.549009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c49c2-90aa-46c6-817b-536787e69ad9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:32.550653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c49c2-90aa-46c6-817b-536787e69ad9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:32.557755 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 387/771] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 23 06:00:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:32.563788 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea42039-b280-487f-9c9e-b7daa59d9970 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:32.572074 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4a9de-de52-4e8b-8a10-943df4b7a4c9 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:32.572360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4a9de-de52-4e8b-8a10-943df4b7a4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:32.572635 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4a9de-de52-4e8b-8a10-943df4b7a4c9 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:32.573963 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 388/772] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:32.580157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9fd94-a255-4d22-9a23-d9df70e3a995 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:32.580445 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9fd94-a255-4d22-9a23-d9df70e3a995 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:32.580717 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9fd94-a255-4d22-9a23-d9df70e3a995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:32.601076 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-02e9fd94-a255-4d22-9a23-d9df70e3a995 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:32.602279 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-02e9fd94-a255-4d22-9a23-d9df70e3a995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:32.727537 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 389/773] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:00:32.741537 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea42039-b280-487f-9c9e-b7daa59d9970 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:32.771993 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa798a2-81ed-417f-9fe8-69ea95d62342 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:32.772323 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa798a2-81ed-417f-9fe8-69ea95d62342 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:32.772599 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa798a2-81ed-417f-9fe8-69ea95d62342 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:32.774093 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 390/774] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:00:32.779615 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6088d-ab3f-48ad-b199-175eb4faeee5 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:32.780098 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6088d-ab3f-48ad-b199-175eb4faeee5 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:32.780576 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6088d-ab3f-48ad-b199-175eb4faeee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:32.810209 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827f66d-d9eb-46a3-9327-9b9a8a68cfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:32.810842 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827f66d-d9eb-46a3-9327-9b9a8a68cfd2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:32.811121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827f66d-d9eb-46a3-9327-9b9a8a68cfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:32.811448 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827f66d-d9eb-46a3-9327-9b9a8a68cfd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:32.812241 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f827f66d-d9eb-46a3-9327-9b9a8a68cfd2 None admin] RBAC: Adding query filter params (interface=public, service_id=f19b42f8acdf411784f922eafd9db44b, region_id=RegionOne) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:32.812599 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f827f66d-d9eb-46a3-9327-9b9a8a68cfd2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:32.813992 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f827f66d-d9eb-46a3-9327-9b9a8a68cfd2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:32.820344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d3f6088d-ab3f-48ad-b199-175eb4faeee5 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:32.821546 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d3f6088d-ab3f-48ad-b199-175eb4faeee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:32.831388 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 385/775] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 23 06:00:32 2022] GET /identity/v3/endpoints?service_id=f19b42f8acdf411784f922eafd9db44b&interface=public®ion_id=RegionOne => generated 194 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:32.837119 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827f66d-d9eb-46a3-9327-9b9a8a68cfd2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:32.842693 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:32.921575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:32.960788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 391/776] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:00:33.016248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827f66d-d9eb-46a3-9327-9b9a8a68cfd2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:33.052422 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a546746-aa14-49e1-bad8-9b32148c39f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:33.053021 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a546746-aa14-49e1-bad8-9b32148c39f3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:33.053341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a546746-aa14-49e1-bad8-9b32148c39f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:33.053610 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a546746-aa14-49e1-bad8-9b32148c39f3 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:33.054209 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a546746-aa14-49e1-bad8-9b32148c39f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:33.055613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a546746-aa14-49e1-bad8-9b32148c39f3 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:33.061518 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 386/777] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 23 06:00:32 2022] GET /identity/v3/services => generated 772 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:33.165902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6088d-ab3f-48ad-b199-175eb4faeee5 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:33.276849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6088d-ab3f-48ad-b199-175eb4faeee5 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:33.287127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a546746-aa14-49e1-bad8-9b32148c39f3 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:33.316803 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298e28e-0a16-4444-a744-7cd496389d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:33.317373 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298e28e-0a16-4444-a744-7cd496389d11 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:33.317682 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298e28e-0a16-4444-a744-7cd496389d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:33.318596 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298e28e-0a16-4444-a744-7cd496389d11 None admin] PATH_INFO: `/v3/services/network` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:33.319192 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298e28e-0a16-4444-a744-7cd496389d11 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:33.320687 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298e28e-0a16-4444-a744-7cd496389d11 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-4298e28e-0a16-4444-a744-7cd496389d11 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:33.331663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:33.332677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:33.333496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:33.333496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:33.333496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:33.333496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:33.333496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:33.333496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:33.333496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:33.333496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:33.333496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:33.333496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:33.333496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:33.333496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:33.333496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 06:00:33.333496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:33.334245 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 392/778] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:33 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:33.339927 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298e28e-0a16-4444-a744-7cd496389d11 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:33.373618 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:33.396259 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:33.419005 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a546746-aa14-49e1-bad8-9b32148c39f3 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:33.456289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298e28e-0a16-4444-a744-7cd496389d11 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:33.461859 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1927ab-6055-479d-911d-efd1ad1b1a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:33.462503 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1927ab-6055-479d-911d-efd1ad1b1a89 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:33.463600 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1927ab-6055-479d-911d-efd1ad1b1a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:33.463877 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1927ab-6055-479d-911d-efd1ad1b1a89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:33.488419 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1927ab-6055-479d-911d-efd1ad1b1a89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-6c1927ab-6055-479d-911d-efd1ad1b1a89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:33.497117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:33.499559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:33.501739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:33.501739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:33.501739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:33.501739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:33.501739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:33.501739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:33.501739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:33.501739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:33.501739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:33.501739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:33.501739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:00:33.501739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:33.503386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 387/779] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:00:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:33.513513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1927ab-6055-479d-911d-efd1ad1b1a89 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:33.530073 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b9755-ef39-41fc-9ed7-db09337fde79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:33.531066 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b9755-ef39-41fc-9ed7-db09337fde79 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:33.531548 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b9755-ef39-41fc-9ed7-db09337fde79 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:33.532034 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b9755-ef39-41fc-9ed7-db09337fde79 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:33.532858 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b9755-ef39-41fc-9ed7-db09337fde79 None admin] RBAC: Adding query filter params (name=network) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:33.533367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b9755-ef39-41fc-9ed7-db09337fde79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:33.534951 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b9755-ef39-41fc-9ed7-db09337fde79 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:33.543791 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 393/780] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 23 06:00:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:33.554339 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b9755-ef39-41fc-9ed7-db09337fde79 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:33.651141 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1927ab-6055-479d-911d-efd1ad1b1a89 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:33.683731 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:33.703927 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6728f-cbe1-4493-8841-60b2568b09c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:33.704610 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6728f-cbe1-4493-8841-60b2568b09c2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:33.705323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6728f-cbe1-4493-8841-60b2568b09c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:33.705323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6728f-cbe1-4493-8841-60b2568b09c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:33.710013 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6728f-cbe1-4493-8841-60b2568b09c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:33.710013 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6728f-cbe1-4493-8841-60b2568b09c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:33.710013 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6728f-cbe1-4493-8841-60b2568b09c2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:33.730505 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 388/781] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:00:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:33.737306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6728f-cbe1-4493-8841-60b2568b09c2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:33.756452 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b9755-ef39-41fc-9ed7-db09337fde79 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:33.771241 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:33.827353 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7929c4-6b2b-4f38-bff1-bbe787fe744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:33.828081 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7929c4-6b2b-4f38-bff1-bbe787fe744e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:33.828362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7929c4-6b2b-4f38-bff1-bbe787fe744e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:33.828627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7929c4-6b2b-4f38-bff1-bbe787fe744e None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:33.829253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7929c4-6b2b-4f38-bff1-bbe787fe744e None admin] RBAC: Adding query filter params (type=network) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:33.829650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7929c4-6b2b-4f38-bff1-bbe787fe744e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:33.831064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7929c4-6b2b-4f38-bff1-bbe787fe744e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:33.844955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 394/782] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 23 06:00:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:33.850794 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7929c4-6b2b-4f38-bff1-bbe787fe744e None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:33.852100 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6728f-cbe1-4493-8841-60b2568b09c2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:33.888556 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bd84d-7b64-40c4-ac81-4452674a5b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:33.889196 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bd84d-7b64-40c4-ac81-4452674a5b54 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:33.889473 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bd84d-7b64-40c4-ac81-4452674a5b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:33.889744 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bd84d-7b64-40c4-ac81-4452674a5b54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:33.901490 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02bd84d-7b64-40c4-ac81-4452674a5b54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:33.903483 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02bd84d-7b64-40c4-ac81-4452674a5b54 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-e02bd84d-7b64-40c4-ac81-4452674a5b54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:33.922378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:33.923386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:33.924262 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:33.924262 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:33.924262 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:33.924867 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 389/783] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:33.930839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bd84d-7b64-40c4-ac81-4452674a5b54 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:33.990911 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:34.047258 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bd84d-7b64-40c4-ac81-4452674a5b54 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:34.085406 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a640d-f1cb-4d66-8240-8ed3144558a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:34.086012 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a640d-f1cb-4d66-8240-8ed3144558a5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:34.086292 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a640d-f1cb-4d66-8240-8ed3144558a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:34.086589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a640d-f1cb-4d66-8240-8ed3144558a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:34.087235 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a640d-f1cb-4d66-8240-8ed3144558a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:34.087532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a640d-f1cb-4d66-8240-8ed3144558a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:34.088917 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a640d-f1cb-4d66-8240-8ed3144558a5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:34.107163 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 390/784] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:34.109445 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a640d-f1cb-4d66-8240-8ed3144558a5 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:34.131888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7929c4-6b2b-4f38-bff1-bbe787fe744e None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:34.169829 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e101b-df02-49ff-95cd-67c9f030dfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:34.170376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e101b-df02-49ff-95cd-67c9f030dfce None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:34.170684 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e101b-df02-49ff-95cd-67c9f030dfce None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:34.170955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e101b-df02-49ff-95cd-67c9f030dfce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:34.171663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e101b-df02-49ff-95cd-67c9f030dfce None admin] RBAC: Adding query filter params (interface=public, service_id=23fc73ea9540446f9b68436626146004, region_id=RegionOne) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:34.171970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e101b-df02-49ff-95cd-67c9f030dfce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:34.173342 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e101b-df02-49ff-95cd-67c9f030dfce None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:34.186682 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 395/785] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 23 06:00:33 2022] GET /identity/v3/endpoints?service_id=23fc73ea9540446f9b68436626146004&interface=public®ion_id=RegionOne => generated 194 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:34.188243 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e101b-df02-49ff-95cd-67c9f030dfce None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:34.227181 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:34.305279 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e101b-df02-49ff-95cd-67c9f030dfce None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:34.344256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386f0ae-c1d6-47cc-93ec-26d4f76d41cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:34.344938 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386f0ae-c1d6-47cc-93ec-26d4f76d41cf None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:34.345219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386f0ae-c1d6-47cc-93ec-26d4f76d41cf None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:34.345485 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386f0ae-c1d6-47cc-93ec-26d4f76d41cf None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:34.346090 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386f0ae-c1d6-47cc-93ec-26d4f76d41cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:34.347499 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386f0ae-c1d6-47cc-93ec-26d4f76d41cf None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:34.353551 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 396/786] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 23 06:00:34 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:34.365186 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a640d-f1cb-4d66-8240-8ed3144558a5 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:34.366910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:34.417905 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df6fb4-af52-44e8-9b29-75d26cf8702d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:34.419115 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df6fb4-af52-44e8-9b29-75d26cf8702d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:34.419386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df6fb4-af52-44e8-9b29-75d26cf8702d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:34.419674 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df6fb4-af52-44e8-9b29-75d26cf8702d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:34.430328 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df6fb4-af52-44e8-9b29-75d26cf8702d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:34.431755 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df6fb4-af52-44e8-9b29-75d26cf8702d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-01df6fb4-af52-44e8-9b29-75d26cf8702d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:34.446589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:34.448997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:34.451807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:34.451807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:34.451807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:34.451807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:34.451807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:34.451807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:34.451807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:34.451807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:34.451807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:00:34.451807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:34.451807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 391/787] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:00:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:34.459394 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386f0ae-c1d6-47cc-93ec-26d4f76d41cf None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:34.614981 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c47b8b-111b-4c48-8ac1-880c2936c94f None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:34.615327 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c47b8b-111b-4c48-8ac1-880c2936c94f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:34.615528 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c47b8b-111b-4c48-8ac1-880c2936c94f None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:34.617188 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 392/788] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:34.623875 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f5016-505f-41f1-9a9a-b7146170aa3b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:34.624158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f5016-505f-41f1-9a9a-b7146170aa3b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:34.624427 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f5016-505f-41f1-9a9a-b7146170aa3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:34.647443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1f6f5016-505f-41f1-9a9a-b7146170aa3b None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:34.648687 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1f6f5016-505f-41f1-9a9a-b7146170aa3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:34.738893 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386f0ae-c1d6-47cc-93ec-26d4f76d41cf None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:34.741231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:34.783191 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1a201-1a9a-4491-9f69-b99c536c1ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:34.784294 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1a201-1a9a-4491-9f69-b99c536c1ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:34.784775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1a201-1a9a-4491-9f69-b99c536c1ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:34.785303 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1a201-1a9a-4491-9f69-b99c536c1ee0 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:34.786251 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1a201-1a9a-4491-9f69-b99c536c1ee0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:34.786790 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1a201-1a9a-4491-9f69-b99c536c1ee0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:34.788453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1a201-1a9a-4491-9f69-b99c536c1ee0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:34.810070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 397/789] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 23 06:00:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:34.815543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1a201-1a9a-4491-9f69-b99c536c1ee0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:34.863264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 393/790] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:34.926689 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf4c34-891b-409e-82ae-a9883538869b None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:34.927443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf4c34-891b-409e-82ae-a9883538869b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:34.927932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf4c34-891b-409e-82ae-a9883538869b None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:34.929516 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 394/791] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:00:34.935294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bda970-558f-4061-907f-4c749223710d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:34.935834 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bda970-558f-4061-907f-4c749223710d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:34.936320 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bda970-558f-4061-907f-4c749223710d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:34.967252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-83bda970-558f-4061-907f-4c749223710d None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:34.968870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-83bda970-558f-4061-907f-4c749223710d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:35.000643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1a201-1a9a-4491-9f69-b99c536c1ee0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:35.055157 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 395/792] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:00:35.081019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8855a29-f5cf-49b2-a25b-55c10af20383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:35.081666 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8855a29-f5cf-49b2-a25b-55c10af20383 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:35.081944 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8855a29-f5cf-49b2-a25b-55c10af20383 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:35.082216 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8855a29-f5cf-49b2-a25b-55c10af20383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:35.098263 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8855a29-f5cf-49b2-a25b-55c10af20383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:35.099725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8855a29-f5cf-49b2-a25b-55c10af20383 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-c8855a29-f5cf-49b2-a25b-55c10af20383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:35.122713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:35.123705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:35.125164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:35.125164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:35.125164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:35.125164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 398/793] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:35.129934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bda970-558f-4061-907f-4c749223710d None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:35.181362 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:35.243845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8855a29-f5cf-49b2-a25b-55c10af20383 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:35.257574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:35.320256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bda970-558f-4061-907f-4c749223710d None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:35.382144 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc0294-26c7-4b06-a81e-ab2a3eb4b928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:35.383152 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc0294-26c7-4b06-a81e-ab2a3eb4b928 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:35.383630 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc0294-26c7-4b06-a81e-ab2a3eb4b928 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:35.384631 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc0294-26c7-4b06-a81e-ab2a3eb4b928 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:35.385602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc0294-26c7-4b06-a81e-ab2a3eb4b928 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:35.386200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc0294-26c7-4b06-a81e-ab2a3eb4b928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:35.389007 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc0294-26c7-4b06-a81e-ab2a3eb4b928 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:35.398944 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8855a29-f5cf-49b2-a25b-55c10af20383 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:35.405405 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 396/794] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:35.406571 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc0294-26c7-4b06-a81e-ab2a3eb4b928 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:35.442137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e2a1a-37d5-4f5d-8cd5-f259a70ea981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:35.442925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e2a1a-37d5-4f5d-8cd5-f259a70ea981 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:35.443276 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e2a1a-37d5-4f5d-8cd5-f259a70ea981 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:35.443844 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e2a1a-37d5-4f5d-8cd5-f259a70ea981 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:35.457711 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e2a1a-37d5-4f5d-8cd5-f259a70ea981 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-569e2a1a-37d5-4f5d-8cd5-f259a70ea981 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:35.464253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:35.465699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:35.466710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:35.466710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:35.466710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:35.466710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:35.466710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:35.466710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:35.466710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:35.466710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:35.466710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:35.466710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:35.466710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:00:35.466710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:35.466710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 399/795] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:00:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:35.467884 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e2a1a-37d5-4f5d-8cd5-f259a70ea981 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:35.516781 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:35.552042 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc0294-26c7-4b06-a81e-ab2a3eb4b928 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:35.572780 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:35.590295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ff243-c2e0-4038-93a5-9f98b9c056fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:35.590953 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ff243-c2e0-4038-93a5-9f98b9c056fb None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:35.591227 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ff243-c2e0-4038-93a5-9f98b9c056fb None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:35.591491 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ff243-c2e0-4038-93a5-9f98b9c056fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:35.601701 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ff243-c2e0-4038-93a5-9f98b9c056fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-f91ff243-c2e0-4038-93a5-9f98b9c056fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:35.607166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:35.608129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:35.608962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:35.608962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:35.608962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:35.608962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:35.608962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:35.608962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:35.608962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:35.608962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:35.608962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:35.608962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:35.608962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:35.608962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:35.608962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 397/796] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:35.609641 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ff243-c2e0-4038-93a5-9f98b9c056fb None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:35.612335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e2a1a-37d5-4f5d-8cd5-f259a70ea981 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:35.653507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb32b15-2a18-48ad-9693-3b40f24d0f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:35.654128 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb32b15-2a18-48ad-9693-3b40f24d0f1e None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:35.654401 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb32b15-2a18-48ad-9693-3b40f24d0f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:35.654795 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb32b15-2a18-48ad-9693-3b40f24d0f1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:35.665989 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb32b15-2a18-48ad-9693-3b40f24d0f1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:35.667889 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb32b15-2a18-48ad-9693-3b40f24d0f1e None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:35.681609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-2cb32b15-2a18-48ad-9693-3b40f24d0f1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:35.681609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:35.681609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:35.681609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:35.681609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:35.681609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:35.681609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:35.681609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:35.681609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:35.681609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:35.681609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:35.681609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:35.681609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:35.681609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:35.681609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:35.682854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:35.683815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:35.683815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:35.683815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:35.683815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:35.683815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:35.683815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:35.683815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:35.683815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:35.687932 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 400/797] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:35.689267 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204bd52-366f-4ae6-a596-98034b07c475 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:35.689518 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204bd52-366f-4ae6-a596-98034b07c475 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:35.689787 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204bd52-366f-4ae6-a596-98034b07c475 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:35.695320 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 401/798] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:35.695950 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204bd52-366f-4ae6-a596-98034b07c475 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:35.725996 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ff243-c2e0-4038-93a5-9f98b9c056fb None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:35.764511 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28a170-4b1b-44d2-a3bb-bcfcbd36411d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:35.765119 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28a170-4b1b-44d2-a3bb-bcfcbd36411d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:35.765396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28a170-4b1b-44d2-a3bb-bcfcbd36411d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:35.765665 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28a170-4b1b-44d2-a3bb-bcfcbd36411d None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:35.766331 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28a170-4b1b-44d2-a3bb-bcfcbd36411d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:35.766661 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28a170-4b1b-44d2-a3bb-bcfcbd36411d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:35.768093 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28a170-4b1b-44d2-a3bb-bcfcbd36411d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:35.776668 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 398/799] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:00:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:35.777523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28a170-4b1b-44d2-a3bb-bcfcbd36411d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:35.782482 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:35.809893 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204bd52-366f-4ae6-a596-98034b07c475 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:35.843153 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:35.850588 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0eca4-b475-4cde-a532-e39e97e0baff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:35.851663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0eca4-b475-4cde-a532-e39e97e0baff None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:35.851934 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0eca4-b475-4cde-a532-e39e97e0baff None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:35.852277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0eca4-b475-4cde-a532-e39e97e0baff None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:35.853018 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0eca4-b475-4cde-a532-e39e97e0baff None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:35.853391 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0eca4-b475-4cde-a532-e39e97e0baff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:35.854927 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0eca4-b475-4cde-a532-e39e97e0baff None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:35.865355 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 402/800] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:35.866442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f586a7e-fe25-4e43-9de1-cb2f1aa0bcf3 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:35.866818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f586a7e-fe25-4e43-9de1-cb2f1aa0bcf3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:35.867151 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f586a7e-fe25-4e43-9de1-cb2f1aa0bcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:35.886247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-0f586a7e-fe25-4e43-9de1-cb2f1aa0bcf3 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:35.888840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0f586a7e-fe25-4e43-9de1-cb2f1aa0bcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:35.914430 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28a170-4b1b-44d2-a3bb-bcfcbd36411d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:35.961897 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fa5fd-1329-4e0a-9a25-cc7d70105706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:35.962858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fa5fd-1329-4e0a-9a25-cc7d70105706 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:35.963204 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fa5fd-1329-4e0a-9a25-cc7d70105706 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:35.963540 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fa5fd-1329-4e0a-9a25-cc7d70105706 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:35.964452 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fa5fd-1329-4e0a-9a25-cc7d70105706 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:35.965707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fa5fd-1329-4e0a-9a25-cc7d70105706 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:35.967460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fa5fd-1329-4e0a-9a25-cc7d70105706 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:35.975599 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 403/801] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:35.980850 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 399/802] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:35.981698 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fa5fd-1329-4e0a-9a25-cc7d70105706 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:35.986193 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f586a7e-fe25-4e43-9de1-cb2f1aa0bcf3 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:36.070195 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:36.102323 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f586a7e-fe25-4e43-9de1-cb2f1aa0bcf3 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:36.119899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fa5fd-1329-4e0a-9a25-cc7d70105706 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:36.128775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:36.144541 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396506cd-128a-4c99-9a22-1c31cbcf3731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:36.146075 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396506cd-128a-4c99-9a22-1c31cbcf3731 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:36.146505 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396506cd-128a-4c99-9a22-1c31cbcf3731 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:36.146955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396506cd-128a-4c99-9a22-1c31cbcf3731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:36.161049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396506cd-128a-4c99-9a22-1c31cbcf3731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:36.162095 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6b447-afb4-4f7d-b634-a99b1daaf6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:36.162831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6b447-afb4-4f7d-b634-a99b1daaf6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:36.163109 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396506cd-128a-4c99-9a22-1c31cbcf3731 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:36.163221 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6b447-afb4-4f7d-b634-a99b1daaf6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:36.163607 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6b447-afb4-4f7d-b634-a99b1daaf6d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:36.176972 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6b447-afb4-4f7d-b634-a99b1daaf6d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:36.178542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6b447-afb4-4f7d-b634-a99b1daaf6d4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-396506cd-128a-4c99-9a22-1c31cbcf3731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:36.182032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:36.183202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:36.184840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:36.184840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:36.184840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:36.186773 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 404/803] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:36.187821 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396506cd-128a-4c99-9a22-1c31cbcf3731 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-fcd6b447-afb4-4f7d-b634-a99b1daaf6d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:36.203017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:36.204187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:36.205280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:36.205280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:36.205280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:36.212063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 400/804] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:36.212063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6b447-afb4-4f7d-b634-a99b1daaf6d4 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:36.321058 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396506cd-128a-4c99-9a22-1c31cbcf3731 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:36.334438 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6b447-afb4-4f7d-b634-a99b1daaf6d4 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:36.351487 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:36.358941 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffe9be-c159-4116-9c9f-edf6f42e5c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:36.359615 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffe9be-c159-4116-9c9f-edf6f42e5c4b None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:36.359892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffe9be-c159-4116-9c9f-edf6f42e5c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:36.360163 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffe9be-c159-4116-9c9f-edf6f42e5c4b None admin] PATH_INFO: `/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/a73720a950944fa08bf1276ea78423b4/roles/f6acb6fdca5a44de9ba00b2b0d4d3202` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:36.362063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffe9be-c159-4116-9c9f-edf6f42e5c4b None admin] RBAC: Authorizing `identity:create_grant(project_id=b0b43dead1664475b9dd52113e08d7ed, user_id=a73720a950944fa08bf1276ea78423b4, role_id=f6acb6fdca5a44de9ba00b2b0d4d3202)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:36.363518 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffe9be-c159-4116-9c9f-edf6f42e5c4b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:36.373246 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62d6dc-f35b-4dcd-b80d-d3e3627dee69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:36.374182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62d6dc-f35b-4dcd-b80d-d3e3627dee69 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:36.374467 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62d6dc-f35b-4dcd-b80d-d3e3627dee69 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:36.374787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62d6dc-f35b-4dcd-b80d-d3e3627dee69 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:36.375413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62d6dc-f35b-4dcd-b80d-d3e3627dee69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:36.375717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62d6dc-f35b-4dcd-b80d-d3e3627dee69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:36.377088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62d6dc-f35b-4dcd-b80d-d3e3627dee69 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:36.380730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 405/805] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:00:36 2022] PUT /identity/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/a73720a950944fa08bf1276ea78423b4/roles/f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:00:36.381743 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffe9be-c159-4116-9c9f-edf6f42e5c4b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:36.398744 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 401/806] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:36.399786 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62d6dc-f35b-4dcd-b80d-d3e3627dee69 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:36.440287 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:36.526735 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62d6dc-f35b-4dcd-b80d-d3e3627dee69 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:36.543789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffe9be-c159-4116-9c9f-edf6f42e5c4b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:36.573925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f263dd-30e3-4b19-824e-831342bd356b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:36.575049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f263dd-30e3-4b19-824e-831342bd356b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:36.575337 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f263dd-30e3-4b19-824e-831342bd356b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:36.575606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f263dd-30e3-4b19-824e-831342bd356b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:36.576249 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f263dd-30e3-4b19-824e-831342bd356b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:36.577580 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f263dd-30e3-4b19-824e-831342bd356b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:36.582041 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8671ef-1298-4164-9624-ffa43f15109f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:36.582682 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8671ef-1298-4164-9624-ffa43f15109f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:36.582960 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8671ef-1298-4164-9624-ffa43f15109f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:36.583229 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8671ef-1298-4164-9624-ffa43f15109f None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:36.583868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8671ef-1298-4164-9624-ffa43f15109f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:36.584164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8671ef-1298-4164-9624-ffa43f15109f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:36.585520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8671ef-1298-4164-9624-ffa43f15109f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-57f263dd-30e3-4b19-824e-831342bd356b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:36.587264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:36.588215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:36.589070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:36.589070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:36.589070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:36.589070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:36.589070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:36.589070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:36.589070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:36.589070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:36.589070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:36.589070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:36.589070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:36.589070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:36.589070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 06:00:36.589070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:36.589070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 402/807] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Aug 23 06:00:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:36.589870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f263dd-30e3-4b19-824e-831342bd356b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:36.604663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 406/808] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:36.605550 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8671ef-1298-4164-9624-ffa43f15109f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:36.639747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:36.712379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f263dd-30e3-4b19-824e-831342bd356b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:36.717851 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:36.726024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8671ef-1298-4164-9624-ffa43f15109f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:36.749672 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3b822-8f97-4804-afc1-0ee655225c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:36.750359 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3b822-8f97-4804-afc1-0ee655225c9f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:36.750612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3b822-8f97-4804-afc1-0ee655225c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:36.750877 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3b822-8f97-4804-afc1-0ee655225c9f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:36.759277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3b822-8f97-4804-afc1-0ee655225c9f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:36.760715 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3b822-8f97-4804-afc1-0ee655225c9f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:36.764668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d13c94-2a37-4873-9a8d-78e1023598a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:36.765343 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d13c94-2a37-4873-9a8d-78e1023598a7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:36.765684 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d13c94-2a37-4873-9a8d-78e1023598a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:36.766017 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d13c94-2a37-4873-9a8d-78e1023598a7 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:36.766740 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d13c94-2a37-4873-9a8d-78e1023598a7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:36.767105 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d13c94-2a37-4873-9a8d-78e1023598a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:36.768525 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d13c94-2a37-4873-9a8d-78e1023598a7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-c8e3b822-8f97-4804-afc1-0ee655225c9f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:36.772572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:36.778858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:36.780055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:36.780055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:36.780055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:36.780055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:36.780055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:36.780055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:36.780055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:36.780055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:36.780055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:00:36.780055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:36.780055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 403/809] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 23 06:00:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:36.780055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3b822-8f97-4804-afc1-0ee655225c9f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:36.780850 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 407/810] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 23 06:00:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:36.780850 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7153ab-b150-4d82-9d8e-f3f50b5a2040 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:36.780850 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7153ab-b150-4d82-9d8e-f3f50b5a2040 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:36.780850 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7153ab-b150-4d82-9d8e-f3f50b5a2040 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:36.783162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 408/811] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:36.783162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7153ab-b150-4d82-9d8e-f3f50b5a2040 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:36.888409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:36.892677 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3b822-8f97-4804-afc1-0ee655225c9f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:36.902085 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7153ab-b150-4d82-9d8e-f3f50b5a2040 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:36.936358 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58b058-02ad-4104-a4a0-1c61eed591bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:36.936987 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58b058-02ad-4104-a4a0-1c61eed591bc None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:36.937273 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58b058-02ad-4104-a4a0-1c61eed591bc None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:36.937545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58b058-02ad-4104-a4a0-1c61eed591bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:36.940430 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6182b-f98d-4c28-bd56-cc2dcd8a6056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:36.941237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6182b-f98d-4c28-bd56-cc2dcd8a6056 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:36.941707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6182b-f98d-4c28-bd56-cc2dcd8a6056 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:36.942124 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6182b-f98d-4c28-bd56-cc2dcd8a6056 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:36.942921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6182b-f98d-4c28-bd56-cc2dcd8a6056 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:36.943356 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6182b-f98d-4c28-bd56-cc2dcd8a6056 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:36.944864 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6182b-f98d-4c28-bd56-cc2dcd8a6056 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:36.945991 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58b058-02ad-4104-a4a0-1c61eed591bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:36.947457 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58b058-02ad-4104-a4a0-1c61eed591bc None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:36.962286 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 409/812] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Aug 23 06:00:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:36.962286 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6182b-f98d-4c28-bd56-cc2dcd8a6056 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-6a58b058-02ad-4104-a4a0-1c61eed591bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:36.969033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:36.969989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:36.970970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:36.970970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:36.970970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:36.970970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:36.970970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:36.970970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:36.970970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:36.970970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:36.970970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 06:00:36.970970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:36.975201 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 404/813] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 23 06:00:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:36.977041 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289dbce-f7a4-46db-8440-b1e638264da7 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:36.977549 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289dbce-f7a4-46db-8440-b1e638264da7 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:36.977912 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289dbce-f7a4-46db-8440-b1e638264da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:36.989827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:36.997335 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a289dbce-f7a4-46db-8440-b1e638264da7 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:36.998646 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a289dbce-f7a4-46db-8440-b1e638264da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:37.071701 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6182b-f98d-4c28-bd56-cc2dcd8a6056 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:37.071906 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 405/814] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:37.072670 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289dbce-f7a4-46db-8440-b1e638264da7 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:37.118028 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52effa2-f029-4237-82b5-b5f1200d5337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:37.118993 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52effa2-f029-4237-82b5-b5f1200d5337 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:37.119502 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52effa2-f029-4237-82b5-b5f1200d5337 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:37.119907 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52effa2-f029-4237-82b5-b5f1200d5337 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:37.120764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52effa2-f029-4237-82b5-b5f1200d5337 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:37.121744 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52effa2-f029-4237-82b5-b5f1200d5337 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:37.123780 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52effa2-f029-4237-82b5-b5f1200d5337 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:37.142904 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 410/815] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 23 06:00:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:37.144101 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52effa2-f029-4237-82b5-b5f1200d5337 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:37.197921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:37.210716 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289dbce-f7a4-46db-8440-b1e638264da7 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:37.235586 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:37.250185 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f831c-aeb5-4071-b4c1-b201b6b684c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:37.250911 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f831c-aeb5-4071-b4c1-b201b6b684c8 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:37.251279 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f831c-aeb5-4071-b4c1-b201b6b684c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:37.251641 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f831c-aeb5-4071-b4c1-b201b6b684c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:37.252394 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f831c-aeb5-4071-b4c1-b201b6b684c8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:37.253847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f831c-aeb5-4071-b4c1-b201b6b684c8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:37.259725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52effa2-f029-4237-82b5-b5f1200d5337 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:37.282956 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 406/816] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 23 06:00:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:00:37.284083 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f831c-aeb5-4071-b4c1-b201b6b684c8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:37.308847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bc267-b23a-467e-abce-9e0d22a27e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:37.310251 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bc267-b23a-467e-abce-9e0d22a27e51 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:37.310699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bc267-b23a-467e-abce-9e0d22a27e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:37.311236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bc267-b23a-467e-abce-9e0d22a27e51 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:37.312122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bc267-b23a-467e-abce-9e0d22a27e51 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:37.312564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bc267-b23a-467e-abce-9e0d22a27e51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:37.314549 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bc267-b23a-467e-abce-9e0d22a27e51 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:37.340363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 411/817] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 23 06:00:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:37.343014 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bc267-b23a-467e-abce-9e0d22a27e51 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:37.466294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f831c-aeb5-4071-b4c1-b201b6b684c8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:37.498756 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bc267-b23a-467e-abce-9e0d22a27e51 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:37.516088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a4b0f-9e86-480a-83ef-18323815ca06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:37.517133 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a4b0f-9e86-480a-83ef-18323815ca06 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:37.517915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a4b0f-9e86-480a-83ef-18323815ca06 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:37.518348 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a4b0f-9e86-480a-83ef-18323815ca06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:37.537950 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd65644-c056-489c-aa8e-d5cd73afd4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:37.538640 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd65644-c056-489c-aa8e-d5cd73afd4eb None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:37.538908 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd65644-c056-489c-aa8e-d5cd73afd4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:37.539173 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd65644-c056-489c-aa8e-d5cd73afd4eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:37.540745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:37.543511 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:37.553617 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd65644-c056-489c-aa8e-d5cd73afd4eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:37.555122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd65644-c056-489c-aa8e-d5cd73afd4eb None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-7bd65644-c056-489c-aa8e-d5cd73afd4eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:37.571256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:37.572256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:37.573137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:37.573137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:37.573137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:37.573137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 412/818] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:37.573799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd65644-c056-489c-aa8e-d5cd73afd4eb None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:37.618698 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a4b0f-9e86-480a-83ef-18323815ca06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:37.620543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a4b0f-9e86-480a-83ef-18323815ca06 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:37.703818 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd65644-c056-489c-aa8e-d5cd73afd4eb None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:37.740434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588f0e2-deec-4d6d-89c8-b61da248edf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:37.740983 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588f0e2-deec-4d6d-89c8-b61da248edf3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:37.741257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588f0e2-deec-4d6d-89c8-b61da248edf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:37.741524 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588f0e2-deec-4d6d-89c8-b61da248edf3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:37.742097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588f0e2-deec-4d6d-89c8-b61da248edf3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:37.743798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588f0e2-deec-4d6d-89c8-b61da248edf3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-4588f0e2-deec-4d6d-89c8-b61da248edf3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:37.754405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:37.755440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:37.756346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:37.756346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:37.756346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:37.756346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:37.756346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:37.756346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:37.756346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:37.756346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:37.756346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:37.756346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:37.756346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:37.756346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:37.756346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 06:00:37.756346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:37.756346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 413/819] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:37 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:37.757150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588f0e2-deec-4d6d-89c8-b61da248edf3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:37.765876 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 407/820] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:00:37.765876 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a4b0f-9e86-480a-83ef-18323815ca06 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:37.823236 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:37.823455 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:37.890418 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588f0e2-deec-4d6d-89c8-b61da248edf3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:37.895485 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a4b0f-9e86-480a-83ef-18323815ca06 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:37.932337 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6540ccd-9c10-4fd7-9bb7-58b4d03ff105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:37.933082 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6540ccd-9c10-4fd7-9bb7-58b4d03ff105 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:37.933468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6540ccd-9c10-4fd7-9bb7-58b4d03ff105 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:37.933907 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6540ccd-9c10-4fd7-9bb7-58b4d03ff105 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:37.935309 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6540ccd-9c10-4fd7-9bb7-58b4d03ff105 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:37.935720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6540ccd-9c10-4fd7-9bb7-58b4d03ff105 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:37.936787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f4a5d-7ccc-4409-9384-8f777bd71b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:37.937286 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6540ccd-9c10-4fd7-9bb7-58b4d03ff105 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:37.938038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f4a5d-7ccc-4409-9384-8f777bd71b2d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:37.938450 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f4a5d-7ccc-4409-9384-8f777bd71b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:37.938973 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f4a5d-7ccc-4409-9384-8f777bd71b2d None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:37.939750 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f4a5d-7ccc-4409-9384-8f777bd71b2d None admin] RBAC: Adding query filter params (name=network) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:37.940290 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f4a5d-7ccc-4409-9384-8f777bd71b2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:37.941854 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f4a5d-7ccc-4409-9384-8f777bd71b2d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:37.947427 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 408/821] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 23 06:00:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:37.948411 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f4a5d-7ccc-4409-9384-8f777bd71b2d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:37.958828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 414/822] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:37.959422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6540ccd-9c10-4fd7-9bb7-58b4d03ff105 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:38.062899 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f4a5d-7ccc-4409-9384-8f777bd71b2d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:38.077592 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:38.078977 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6540ccd-9c10-4fd7-9bb7-58b4d03ff105 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:38.091443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:38.108296 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae106bb-3919-4872-80ff-e99e115cbfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:38.109243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae106bb-3919-4872-80ff-e99e115cbfee None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:38.109702 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae106bb-3919-4872-80ff-e99e115cbfee None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:38.110241 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae106bb-3919-4872-80ff-e99e115cbfee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:38.118574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019cdae-7484-46d9-843c-9bca303599b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:38.119088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019cdae-7484-46d9-843c-9bca303599b5 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:38.119369 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019cdae-7484-46d9-843c-9bca303599b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:38.119638 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019cdae-7484-46d9-843c-9bca303599b5 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:38.120263 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019cdae-7484-46d9-843c-9bca303599b5 None admin] RBAC: Adding query filter params (type=network) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:38.120564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019cdae-7484-46d9-843c-9bca303599b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:38.122791 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019cdae-7484-46d9-843c-9bca303599b5 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:38.124079 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae106bb-3919-4872-80ff-e99e115cbfee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:38.125819 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae106bb-3919-4872-80ff-e99e115cbfee None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:38.128111 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 415/823] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 23 06:00:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:38.128706 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019cdae-7484-46d9-843c-9bca303599b5 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-bae106bb-3919-4872-80ff-e99e115cbfee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:38.143488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:38.144472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:38.145352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:38.145352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:38.145352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:38.147282 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 409/824] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:38.148359 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae106bb-3919-4872-80ff-e99e115cbfee None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:38.242369 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019cdae-7484-46d9-843c-9bca303599b5 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:38.260301 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae106bb-3919-4872-80ff-e99e115cbfee None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:38.282544 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e578db-7e9a-44cc-97d5-fdc2152db9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:38.283241 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e578db-7e9a-44cc-97d5-fdc2152db9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:38.283577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e578db-7e9a-44cc-97d5-fdc2152db9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:38.283858 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e578db-7e9a-44cc-97d5-fdc2152db9d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:38.291165 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e578db-7e9a-44cc-97d5-fdc2152db9d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:38.292578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e578db-7e9a-44cc-97d5-fdc2152db9d4 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:38.299855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a4f31-4820-4045-8a0a-787f369461ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:38.300773 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a4f31-4820-4045-8a0a-787f369461ab None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:38.301256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a4f31-4820-4045-8a0a-787f369461ab None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:38.301718 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a4f31-4820-4045-8a0a-787f369461ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:38.302587 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a4f31-4820-4045-8a0a-787f369461ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:38.304136 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a4f31-4820-4045-8a0a-787f369461ab None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:38.306007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-23e578db-7e9a-44cc-97d5-fdc2152db9d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:38.306007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:38.306007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:38.306007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:38.306007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:38.306007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:38.306007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:38.306007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:38.306007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:38.306007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:38.306007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:38.306007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:38.306007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:38.306007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:38.306007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:38.307097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:38.308009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:38.308009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:38.308009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:38.308009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:38.308009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:38.308009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:38.308009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:38.308009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:38.308009 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 416/825] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:38.308489 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e578db-7e9a-44cc-97d5-fdc2152db9d4 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:38.330732 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 410/826] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 23 06:00:38 2022] POST /identity/v3/endpoints => generated 337 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:00:38.332106 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a4f31-4820-4045-8a0a-787f369461ab None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:38.349258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:38.359707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:38.466612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e578db-7e9a-44cc-97d5-fdc2152db9d4 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:38.498808 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a4f31-4820-4045-8a0a-787f369461ab None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:38.524182 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea57405-cdd5-47de-b9dd-f4322b7c2c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:38.524790 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea57405-cdd5-47de-b9dd-f4322b7c2c64 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:38.525064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea57405-cdd5-47de-b9dd-f4322b7c2c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:38.525329 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea57405-cdd5-47de-b9dd-f4322b7c2c64 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:38.525971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea57405-cdd5-47de-b9dd-f4322b7c2c64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:38.526269 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea57405-cdd5-47de-b9dd-f4322b7c2c64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:38.530195 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea57405-cdd5-47de-b9dd-f4322b7c2c64 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:38.549501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 417/827] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:38.554053 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc535d-645a-4f3c-a2e7-2c1aaf46d03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:38.554053 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc535d-645a-4f3c-a2e7-2c1aaf46d03c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:38.554053 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc535d-645a-4f3c-a2e7-2c1aaf46d03c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:38.554053 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc535d-645a-4f3c-a2e7-2c1aaf46d03c None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:38.554379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc535d-645a-4f3c-a2e7-2c1aaf46d03c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:38.554379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc535d-645a-4f3c-a2e7-2c1aaf46d03c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:38.555212 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29286b-806c-416f-beb6-3b5c18f32b55 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:38.555312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc535d-645a-4f3c-a2e7-2c1aaf46d03c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:38.555492 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29286b-806c-416f-beb6-3b5c18f32b55 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:38.555760 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29286b-806c-416f-beb6-3b5c18f32b55 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:38.557859 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 418/828] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:38.558699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29286b-806c-416f-beb6-3b5c18f32b55 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:38.570588 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 411/829] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:38.572113 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa2dc4-0bb4-49b7-94e7-c341555cb602 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:38.572636 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa2dc4-0bb4-49b7-94e7-c341555cb602 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:38.573168 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa2dc4-0bb4-49b7-94e7-c341555cb602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:38.598673 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-15aa2dc4-0bb4-49b7-94e7-c341555cb602 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:38.600268 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-15aa2dc4-0bb4-49b7-94e7-c341555cb602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:38.679194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:38.694538 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 412/830] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:38.694538 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa2dc4-0bb4-49b7-94e7-c341555cb602 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:38.696516 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29286b-806c-416f-beb6-3b5c18f32b55 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:38.697123 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:38.733802 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cb3aa-3dd0-4d41-8db1-3c2698fde722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:38.734619 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cb3aa-3dd0-4d41-8db1-3c2698fde722 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:38.734907 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cb3aa-3dd0-4d41-8db1-3c2698fde722 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:38.735241 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cb3aa-3dd0-4d41-8db1-3c2698fde722 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:38.742500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cb3aa-3dd0-4d41-8db1-3c2698fde722 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:38.744329 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cb3aa-3dd0-4d41-8db1-3c2698fde722 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:38.756053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-c93cb3aa-3dd0-4d41-8db1-3c2698fde722 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:38.756053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:38.756053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:38.756053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:38.756053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:38.756053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:38.756053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:38.756053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:38.756053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:38.756053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:38.756053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:38.756053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:38.756053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:38.756053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:38.756053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:38.757054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:38.763883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:38.763883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:38.763883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:38.763883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:38.763883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:38.763883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:38.763883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:38.763883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:38.763883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 419/831] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:38.764644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cb3aa-3dd0-4d41-8db1-3c2698fde722 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:38.810276 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa2dc4-0bb4-49b7-94e7-c341555cb602 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:38.846141 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bd975-f655-4e43-bbf9-ea52a9becf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:38.846850 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bd975-f655-4e43-bbf9-ea52a9becf58 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:38.847127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bd975-f655-4e43-bbf9-ea52a9becf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:38.847401 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bd975-f655-4e43-bbf9-ea52a9becf58 None admin] PATH_INFO: `/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/078a012fea464954b0632a6ce86d7cec/roles/3209a3d044b84a0ebd7cf1edf2da7120` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:38.863298 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bd975-f655-4e43-bbf9-ea52a9becf58 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0b43dead1664475b9dd52113e08d7ed, user_id=078a012fea464954b0632a6ce86d7cec, role_id=3209a3d044b84a0ebd7cf1edf2da7120)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:38.864669 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bd975-f655-4e43-bbf9-ea52a9becf58 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:38.877642 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cb3aa-3dd0-4d41-8db1-3c2698fde722 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:38.886621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 413/832] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:00:38 2022] PUT /identity/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/078a012fea464954b0632a6ce86d7cec/roles/3209a3d044b84a0ebd7cf1edf2da7120 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:00:38.920360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a5c7a-ffb6-4860-9c62-985b92e143ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:38.921518 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a5c7a-ffb6-4860-9c62-985b92e143ab None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:38.922142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a5c7a-ffb6-4860-9c62-985b92e143ab None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:38.922644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a5c7a-ffb6-4860-9c62-985b92e143ab None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:38.923697 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a5c7a-ffb6-4860-9c62-985b92e143ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:38.924235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a5c7a-ffb6-4860-9c62-985b92e143ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:38.926121 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a5c7a-ffb6-4860-9c62-985b92e143ab None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:38.942059 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 420/833] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:38.954967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bd975-f655-4e43-bbf9-ea52a9becf58 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:38.986963 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a5c7a-ffb6-4860-9c62-985b92e143ab None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:39.008681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:39.034244 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:39.099083 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bd975-f655-4e43-bbf9-ea52a9becf58 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:39.136004 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a5c7a-ffb6-4860-9c62-985b92e143ab None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:39.146190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f8cc1-484e-4a74-9ca1-7954c4064ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:39.146830 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f8cc1-484e-4a74-9ca1-7954c4064ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:39.147107 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f8cc1-484e-4a74-9ca1-7954c4064ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:39.147377 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f8cc1-484e-4a74-9ca1-7954c4064ff7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:39.148476 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f8cc1-484e-4a74-9ca1-7954c4064ff7 None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=91267a049343452eb208205ee3ea27c4) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:39.148792 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f8cc1-484e-4a74-9ca1-7954c4064ff7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:39.150956 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f8cc1-484e-4a74-9ca1-7954c4064ff7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:39.159490 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 414/834] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:38 2022] GET /identity/v3/role_assignments?user.id=91267a049343452eb208205ee3ea27c4&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:39.178654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5b981-3e4f-4710-8530-4334eb7ac3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:39.179306 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5b981-3e4f-4710-8530-4334eb7ac3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:39.179590 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5b981-3e4f-4710-8530-4334eb7ac3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:39.179860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5b981-3e4f-4710-8530-4334eb7ac3a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:39.193944 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5b981-3e4f-4710-8530-4334eb7ac3a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-c6c5b981-3e4f-4710-8530-4334eb7ac3a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:39.199024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:39.200591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:39.201453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:39.201453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:39.201453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:39.201453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:39.201453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:39.201453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:39.201453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:39.201453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:39.201453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:39.201453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:39.201453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:00:39.201453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:39.201453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 421/835] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:00:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:39.208788 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f8cc1-484e-4a74-9ca1-7954c4064ff7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:39.328117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f8cc1-484e-4a74-9ca1-7954c4064ff7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:39.349851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:39.369546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe735d3-2dc4-47d5-9191-1bd360ce01b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:39.370165 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe735d3-2dc4-47d5-9191-1bd360ce01b4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:39.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe735d3-2dc4-47d5-9191-1bd360ce01b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:39.370745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe735d3-2dc4-47d5-9191-1bd360ce01b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:39.371526 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe735d3-2dc4-47d5-9191-1bd360ce01b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:39.372098 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe735d3-2dc4-47d5-9191-1bd360ce01b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:39.373084 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe735d3-2dc4-47d5-9191-1bd360ce01b4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:39.383705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 415/836] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:00:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:39.390263 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5b981-3e4f-4710-8530-4334eb7ac3a0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:39.466355 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479a62d-0c3f-4a5f-8721-84fcbcb74b00 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:39.466714 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479a62d-0c3f-4a5f-8721-84fcbcb74b00 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:39.466968 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479a62d-0c3f-4a5f-8721-84fcbcb74b00 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:39.468452 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 416/837] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:00:39.474086 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47781beb-a154-46e9-80c6-288568ec95d8 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:39.474682 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47781beb-a154-46e9-80c6-288568ec95d8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:39.475155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47781beb-a154-46e9-80c6-288568ec95d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:39.483162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:39.495727 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-47781beb-a154-46e9-80c6-288568ec95d8 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:39.498307 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-47781beb-a154-46e9-80c6-288568ec95d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:39.510885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5b981-3e4f-4710-8530-4334eb7ac3a0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:39.554933 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14180d65-6d80-4d66-92c8-2ab755bbc459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:39.555564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14180d65-6d80-4d66-92c8-2ab755bbc459 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:39.555831 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14180d65-6d80-4d66-92c8-2ab755bbc459 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:39.558547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14180d65-6d80-4d66-92c8-2ab755bbc459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:39.567654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14180d65-6d80-4d66-92c8-2ab755bbc459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:39.569099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14180d65-6d80-4d66-92c8-2ab755bbc459 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-14180d65-6d80-4d66-92c8-2ab755bbc459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:39.584827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:39.585810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:39.586729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:39.586729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:39.586729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:39.586729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 422/838] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:39.595540 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14180d65-6d80-4d66-92c8-2ab755bbc459 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:39.596785 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 417/839] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:00:39.708670 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14180d65-6d80-4d66-92c8-2ab755bbc459 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:39.743935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:39.754349 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a38da-882f-4b95-b07b-8e6aa8da2e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:39.754990 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a38da-882f-4b95-b07b-8e6aa8da2e7f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:39.755275 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a38da-882f-4b95-b07b-8e6aa8da2e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:39.755551 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a38da-882f-4b95-b07b-8e6aa8da2e7f None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:39.756199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a38da-882f-4b95-b07b-8e6aa8da2e7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:39.756505 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a38da-882f-4b95-b07b-8e6aa8da2e7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:39.758129 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a38da-882f-4b95-b07b-8e6aa8da2e7f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:39.770736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 423/840] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:39.775909 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47781beb-a154-46e9-80c6-288568ec95d8 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:39.881697 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47781beb-a154-46e9-80c6-288568ec95d8 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:39.883397 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a38da-882f-4b95-b07b-8e6aa8da2e7f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:39.899281 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:39.919840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97357a-9e5e-4760-971c-2a16c4f0386d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:39.920410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97357a-9e5e-4760-971c-2a16c4f0386d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:39.920688 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97357a-9e5e-4760-971c-2a16c4f0386d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:39.920951 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97357a-9e5e-4760-971c-2a16c4f0386d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:39.928931 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97357a-9e5e-4760-971c-2a16c4f0386d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:39.930383 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97357a-9e5e-4760-971c-2a16c4f0386d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-dd97357a-9e5e-4760-971c-2a16c4f0386d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:39.944297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:39.945323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:39.946919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:39.946919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:39.946919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:39.946919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:39.946919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:39.946919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:39.946919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:39.946919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:39.946919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 06:00:39.946919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:39.946919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 418/841] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:00:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:39.951540 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97357a-9e5e-4760-971c-2a16c4f0386d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:40.013878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a38da-882f-4b95-b07b-8e6aa8da2e7f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:40.051824 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e476c1d-c593-4ed7-a75c-557247ae3369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:40.052794 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e476c1d-c593-4ed7-a75c-557247ae3369 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:40.053290 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e476c1d-c593-4ed7-a75c-557247ae3369 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:40.053659 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e476c1d-c593-4ed7-a75c-557247ae3369 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:40.054408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e476c1d-c593-4ed7-a75c-557247ae3369 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:40.055987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e476c1d-c593-4ed7-a75c-557247ae3369 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-6e476c1d-c593-4ed7-a75c-557247ae3369 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:40.067150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:40.068325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:40.069160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:40.069160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:40.069160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:40.069160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:40.069160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:40.069160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:40.069160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:40.069160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:40.069160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:40.069160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:40.069160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:40.069160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:40.069160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 06:00:40.069160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:40.070273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 424/842] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 06:00:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:40.072937 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97357a-9e5e-4760-971c-2a16c4f0386d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:40.075001 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e476c1d-c593-4ed7-a75c-557247ae3369 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:40.116351 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220539ae-c731-4a35-9737-7e7392a50c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:40.117076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220539ae-c731-4a35-9737-7e7392a50c29 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:40.117353 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220539ae-c731-4a35-9737-7e7392a50c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:40.117625 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220539ae-c731-4a35-9737-7e7392a50c29 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:40.118896 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220539ae-c731-4a35-9737-7e7392a50c29 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:40.119251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220539ae-c731-4a35-9737-7e7392a50c29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:40.122134 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220539ae-c731-4a35-9737-7e7392a50c29 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:40.136866 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:40.156473 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 419/843] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Aug 23 06:00:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:40.157837 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220539ae-c731-4a35-9737-7e7392a50c29 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:40.196152 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:40.207098 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e476c1d-c593-4ed7-a75c-557247ae3369 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:40.244316 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a231c-5fb0-4a01-a5af-dee32d30154a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:40.245235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a231c-5fb0-4a01-a5af-dee32d30154a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:40.245666 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a231c-5fb0-4a01-a5af-dee32d30154a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:40.245942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a231c-5fb0-4a01-a5af-dee32d30154a None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:40.246923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a231c-5fb0-4a01-a5af-dee32d30154a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:40.247303 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a231c-5fb0-4a01-a5af-dee32d30154a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:40.248785 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a231c-5fb0-4a01-a5af-dee32d30154a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:40.258669 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 425/844] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 06:00:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:40.259988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a231c-5fb0-4a01-a5af-dee32d30154a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:40.289490 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220539ae-c731-4a35-9737-7e7392a50c29 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:40.333614 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5145c-4cb2-4c4a-9820-ec63edfbde01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:40.334706 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5145c-4cb2-4c4a-9820-ec63edfbde01 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:40.335013 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5145c-4cb2-4c4a-9820-ec63edfbde01 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:40.335396 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5145c-4cb2-4c4a-9820-ec63edfbde01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:40.384652 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a231c-5fb0-4a01-a5af-dee32d30154a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:40.401932 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:40.427170 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afdaaa-976f-4064-878c-c5e138c0f7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:40.428211 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afdaaa-976f-4064-878c-c5e138c0f7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:40.428714 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afdaaa-976f-4064-878c-c5e138c0f7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:40.429184 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afdaaa-976f-4064-878c-c5e138c0f7d0 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:40.430010 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afdaaa-976f-4064-878c-c5e138c0f7d0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:40.430594 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afdaaa-976f-4064-878c-c5e138c0f7d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:40.432422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afdaaa-976f-4064-878c-c5e138c0f7d0 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:40.433761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5145c-4cb2-4c4a-9820-ec63edfbde01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:40.436756 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5145c-4cb2-4c4a-9820-ec63edfbde01 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:40.438906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 426/845] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 06:00:40 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:40.470712 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:40.558038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 420/846] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 06:00:40.564510 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afdaaa-976f-4064-878c-c5e138c0f7d0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:40.676962 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afdaaa-976f-4064-878c-c5e138c0f7d0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:40.712956 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29c70a-784b-4024-9601-5b3f57a0285a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:40.713934 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29c70a-784b-4024-9601-5b3f57a0285a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:40.714457 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29c70a-784b-4024-9601-5b3f57a0285a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:40.715503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29c70a-784b-4024-9601-5b3f57a0285a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:40.726634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29c70a-784b-4024-9601-5b3f57a0285a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:40.728421 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29c70a-784b-4024-9601-5b3f57a0285a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-da29c70a-784b-4024-9601-5b3f57a0285a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:40.743688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:40.744665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:40.745568 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:40.745568 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:40.745568 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:40.748195 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 427/847] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:40.754049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5145c-4cb2-4c4a-9820-ec63edfbde01 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:40.867426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5145c-4cb2-4c4a-9820-ec63edfbde01 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:40.897125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:40.907786 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f2013-d4a6-4a5e-98ad-0d09b1a63f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:40.908621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f2013-d4a6-4a5e-98ad-0d09b1a63f99 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:40.908902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f2013-d4a6-4a5e-98ad-0d09b1a63f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:40.909177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f2013-d4a6-4a5e-98ad-0d09b1a63f99 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:40.909830 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f2013-d4a6-4a5e-98ad-0d09b1a63f99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:40.910138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f2013-d4a6-4a5e-98ad-0d09b1a63f99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:40.911559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f2013-d4a6-4a5e-98ad-0d09b1a63f99 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:40.925779 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 421/848] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:40.931518 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29c70a-784b-4024-9601-5b3f57a0285a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:40.967545 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:41.020218 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6c05f-687e-4135-91fa-1fb7b6d1d0be None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:41.020570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6c05f-687e-4135-91fa-1fb7b6d1d0be None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:41.020783 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6c05f-687e-4135-91fa-1fb7b6d1d0be None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:41.022269 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 422/849] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:41.027576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac508a-9c01-4c64-98da-f862acd81cf3 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:41.028507 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac508a-9c01-4c64-98da-f862acd81cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:41.028793 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac508a-9c01-4c64-98da-f862acd81cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:41.048616 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-58ac508a-9c01-4c64-98da-f862acd81cf3 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:41.049820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-58ac508a-9c01-4c64-98da-f862acd81cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:41.050582 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29c70a-784b-4024-9601-5b3f57a0285a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:41.087131 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6427c88-49e6-466c-baf5-339d98f9a069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:41.088271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6427c88-49e6-466c-baf5-339d98f9a069 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:41.088848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6427c88-49e6-466c-baf5-339d98f9a069 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:41.089316 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6427c88-49e6-466c-baf5-339d98f9a069 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:41.097081 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6427c88-49e6-466c-baf5-339d98f9a069 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:41.098947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6427c88-49e6-466c-baf5-339d98f9a069 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:41.112084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-f6427c88-49e6-466c-baf5-339d98f9a069 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:41.112084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:41.112084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:41.112084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:41.112084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:41.112084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:41.112084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:41.112084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:41.112084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:41.112084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:41.112084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:41.112084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:41.112084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:41.112084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:41.112084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:41.113035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:41.113915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:41.113915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:41.113915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:41.113915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:41.113915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:41.113915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:41.113915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:41.113915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:41.116465 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 428/850] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:41.122758 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6427c88-49e6-466c-baf5-339d98f9a069 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:41.134627 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 423/851] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:41.220420 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:41.229667 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6427c88-49e6-466c-baf5-339d98f9a069 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:41.266763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5209fc-ce28-4ac2-b42e-9ed0eacfcda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:41.267367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5209fc-ce28-4ac2-b42e-9ed0eacfcda6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:41.267640 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5209fc-ce28-4ac2-b42e-9ed0eacfcda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:41.267907 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5209fc-ce28-4ac2-b42e-9ed0eacfcda6 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:41.268592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5209fc-ce28-4ac2-b42e-9ed0eacfcda6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:41.268892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5209fc-ce28-4ac2-b42e-9ed0eacfcda6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:41.270319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5209fc-ce28-4ac2-b42e-9ed0eacfcda6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:41.283762 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 429/852] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:41.289838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac508a-9c01-4c64-98da-f862acd81cf3 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:41.329667 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa8fbd-07ff-40b5-8b57-f9bfdf98243c None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:41.329986 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa8fbd-07ff-40b5-8b57-f9bfdf98243c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:41.330202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa8fbd-07ff-40b5-8b57-f9bfdf98243c None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:41.331990 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 430/853] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:00:41.336949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c80c3-0ddd-42ed-995b-da427703b35d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:41.337230 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c80c3-0ddd-42ed-995b-da427703b35d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:41.337605 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c80c3-0ddd-42ed-995b-da427703b35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:41.357296 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ed9c80c3-0ddd-42ed-995b-da427703b35d None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:41.358486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ed9c80c3-0ddd-42ed-995b-da427703b35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:41.407661 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac508a-9c01-4c64-98da-f862acd81cf3 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:41.426902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:41.436799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 431/854] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:41.437657 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c80c3-0ddd-42ed-995b-da427703b35d None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:41.451612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8eade-f9b2-4e3c-9fd5-3a865af4f226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:41.452848 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8eade-f9b2-4e3c-9fd5-3a865af4f226 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:41.453864 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8eade-f9b2-4e3c-9fd5-3a865af4f226 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:41.454491 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8eade-f9b2-4e3c-9fd5-3a865af4f226 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:41.455525 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8eade-f9b2-4e3c-9fd5-3a865af4f226 None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=a73720a950944fa08bf1276ea78423b4) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:41.456250 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8eade-f9b2-4e3c-9fd5-3a865af4f226 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:41.458300 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8eade-f9b2-4e3c-9fd5-3a865af4f226 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:41.468627 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 424/855] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:41 2022] GET /identity/v3/role_assignments?user.id=a73720a950944fa08bf1276ea78423b4&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:41.535352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:41.559133 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c80c3-0ddd-42ed-995b-da427703b35d None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:41.601341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b55005-b67d-414f-98cb-e3a4cbd2086f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:41.601977 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b55005-b67d-414f-98cb-e3a4cbd2086f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:41.602260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b55005-b67d-414f-98cb-e3a4cbd2086f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:41.602543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b55005-b67d-414f-98cb-e3a4cbd2086f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:41.615415 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b55005-b67d-414f-98cb-e3a4cbd2086f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-34b55005-b67d-414f-98cb-e3a4cbd2086f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:41.620350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:41.621952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:41.623561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:41.623561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:41.623561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:41.623561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:41.623561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:41.623561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:41.623561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:41.623561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:41.623561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:41.623561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:41.623561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:00:41.623561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:41.623561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 432/856] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:00:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:41.628640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8eade-f9b2-4e3c-9fd5-3a865af4f226 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:41.723757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b55005-b67d-414f-98cb-e3a4cbd2086f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:41.741729 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8eade-f9b2-4e3c-9fd5-3a865af4f226 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:41.784355 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23854984-044a-4f52-bde3-e9d28720da61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:41.784969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23854984-044a-4f52-bde3-e9d28720da61 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:41.785249 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23854984-044a-4f52-bde3-e9d28720da61 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:41.785522 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23854984-044a-4f52-bde3-e9d28720da61 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:41.786663 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23854984-044a-4f52-bde3-e9d28720da61 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:41.786975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23854984-044a-4f52-bde3-e9d28720da61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:41.788661 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23854984-044a-4f52-bde3-e9d28720da61 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:41.800408 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 425/857] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:00:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:41.805736 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23854984-044a-4f52-bde3-e9d28720da61 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:41.847406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b55005-b67d-414f-98cb-e3a4cbd2086f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:41.882910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:41.891660 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415dba5d-348c-4d72-88e9-b594573bf461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:41.892303 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dba5d-348c-4d72-88e9-b594573bf461 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:41.892581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dba5d-348c-4d72-88e9-b594573bf461 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:41.892852 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dba5d-348c-4d72-88e9-b594573bf461 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:41.893435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415dba5d-348c-4d72-88e9-b594573bf461 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:41.894863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415dba5d-348c-4d72-88e9-b594573bf461 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-415dba5d-348c-4d72-88e9-b594573bf461 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:41.911289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:41.912344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:41.918848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:41.918848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:41.918848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:41.918848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:41.918848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:41.918848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:41.918848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:41.918848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:41.918848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:41.918848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:41.918848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:41.918848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:41.918848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 06:00:41.918848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:41.918848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 433/858] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Aug 23 06:00:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:41.920924 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415dba5d-348c-4d72-88e9-b594573bf461 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:41.927028 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:41.938814 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23854984-044a-4f52-bde3-e9d28720da61 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:41.990083 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c5139-0d0b-4e17-8732-af85a631332a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:41.991074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c5139-0d0b-4e17-8732-af85a631332a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:41.991369 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c5139-0d0b-4e17-8732-af85a631332a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:41.991641 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c5139-0d0b-4e17-8732-af85a631332a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:42.006620 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c5139-0d0b-4e17-8732-af85a631332a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:42.008747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c5139-0d0b-4e17-8732-af85a631332a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-a43c5139-0d0b-4e17-8732-af85a631332a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:42.027915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:42.029200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:42.030424 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:42.030424 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:42.030424 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:42.034182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 426/859] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:42.035182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c5139-0d0b-4e17-8732-af85a631332a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:42.065976 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415dba5d-348c-4d72-88e9-b594573bf461 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:42.112554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913e0a8-be88-4421-994c-6f40d47ec838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:42.113758 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913e0a8-be88-4421-994c-6f40d47ec838 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:42.114242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913e0a8-be88-4421-994c-6f40d47ec838 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:42.114854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913e0a8-be88-4421-994c-6f40d47ec838 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:42.116063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913e0a8-be88-4421-994c-6f40d47ec838 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:42.116653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913e0a8-be88-4421-994c-6f40d47ec838 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:42.118739 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913e0a8-be88-4421-994c-6f40d47ec838 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:42.130234 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 434/860] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 23 06:00:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:42.131922 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913e0a8-be88-4421-994c-6f40d47ec838 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:42.177118 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:42.183338 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c5139-0d0b-4e17-8732-af85a631332a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:42.220101 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2db14-4b23-4f7d-b5cf-def00bc0f372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:42.220885 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2db14-4b23-4f7d-b5cf-def00bc0f372 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:42.221276 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2db14-4b23-4f7d-b5cf-def00bc0f372 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:42.221635 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2db14-4b23-4f7d-b5cf-def00bc0f372 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:42.222368 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2db14-4b23-4f7d-b5cf-def00bc0f372 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:42.222806 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2db14-4b23-4f7d-b5cf-def00bc0f372 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:42.223714 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:42.224806 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2db14-4b23-4f7d-b5cf-def00bc0f372 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:42.238551 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 427/861] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:42.244341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2db14-4b23-4f7d-b5cf-def00bc0f372 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:42.269081 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913e0a8-be88-4421-994c-6f40d47ec838 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:42.305843 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2585c-1ea3-4f9d-b8e1-15924d58f64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:42.306454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2585c-1ea3-4f9d-b8e1-15924d58f64f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:42.306773 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2585c-1ea3-4f9d-b8e1-15924d58f64f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:42.307045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2585c-1ea3-4f9d-b8e1-15924d58f64f None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:42.307673 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2585c-1ea3-4f9d-b8e1-15924d58f64f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:42.307971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2585c-1ea3-4f9d-b8e1-15924d58f64f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:42.310200 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2585c-1ea3-4f9d-b8e1-15924d58f64f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:42.320624 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 435/862] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 23 06:00:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:42.376211 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2db14-4b23-4f7d-b5cf-def00bc0f372 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:42.419298 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166ae3e-671f-43ee-81cf-1d11ce65a2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:42.420045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166ae3e-671f-43ee-81cf-1d11ce65a2cc None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:42.420318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166ae3e-671f-43ee-81cf-1d11ce65a2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:42.420584 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166ae3e-671f-43ee-81cf-1d11ce65a2cc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:42.428814 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166ae3e-671f-43ee-81cf-1d11ce65a2cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:42.430265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166ae3e-671f-43ee-81cf-1d11ce65a2cc None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-7166ae3e-671f-43ee-81cf-1d11ce65a2cc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:42.443598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:42.444632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:42.445559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:42.445559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:42.445559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:42.445559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:42.445559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:42.445559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:42.445559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:42.445559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:42.445559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 06:00:42.445559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:42.445559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 428/863] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 23 06:00:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:42.450842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2585c-1ea3-4f9d-b8e1-15924d58f64f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:42.499819 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9271453-676f-479f-9df7-1c86e6f48213 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:42.500171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9271453-676f-479f-9df7-1c86e6f48213 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:42.500444 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9271453-676f-479f-9df7-1c86e6f48213 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:42.502489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 429/864] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:00:42.508164 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14321b99-bf4d-4512-ae12-aa2ea708f576 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:42.508443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14321b99-bf4d-4512-ae12-aa2ea708f576 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:42.508709 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14321b99-bf4d-4512-ae12-aa2ea708f576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:42.531729 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-14321b99-bf4d-4512-ae12-aa2ea708f576 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:42.532920 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-14321b99-bf4d-4512-ae12-aa2ea708f576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:42.546592 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:42.569201 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2585c-1ea3-4f9d-b8e1-15924d58f64f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:42.600914 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:42.607021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059674c-98dd-436a-b490-37f431ca520a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:42.608543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059674c-98dd-436a-b490-37f431ca520a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:42.608940 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059674c-98dd-436a-b490-37f431ca520a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:42.609311 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059674c-98dd-436a-b490-37f431ca520a None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:42.610171 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059674c-98dd-436a-b490-37f431ca520a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:42.610570 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059674c-98dd-436a-b490-37f431ca520a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:42.612192 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059674c-98dd-436a-b490-37f431ca520a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:42.623874 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 430/865] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:00:42.632514 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 436/866] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 23 06:00:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:42.637943 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14321b99-bf4d-4512-ae12-aa2ea708f576 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:42.742173 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14321b99-bf4d-4512-ae12-aa2ea708f576 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:42.778021 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303741a-6b1f-436e-8696-1d8d565110f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:42.778911 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303741a-6b1f-436e-8696-1d8d565110f0 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:42.779326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303741a-6b1f-436e-8696-1d8d565110f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:42.779728 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303741a-6b1f-436e-8696-1d8d565110f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:42.799602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:42.857397 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303741a-6b1f-436e-8696-1d8d565110f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:42.858892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303741a-6b1f-436e-8696-1d8d565110f0 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:42.896228 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059674c-98dd-436a-b490-37f431ca520a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:42.992310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 431/867] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:42 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:00:42.998401 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303741a-6b1f-436e-8696-1d8d565110f0 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:43.022829 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059674c-98dd-436a-b490-37f431ca520a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:43.052105 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:43.068975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378443ab-fd05-4b96-9506-de6f108f8b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:43.069583 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378443ab-fd05-4b96-9506-de6f108f8b76 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:43.069856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378443ab-fd05-4b96-9506-de6f108f8b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:43.070276 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378443ab-fd05-4b96-9506-de6f108f8b76 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:43.070945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378443ab-fd05-4b96-9506-de6f108f8b76 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:43.072913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378443ab-fd05-4b96-9506-de6f108f8b76 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:43.094640 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 437/868] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 23 06:00:42 2022] POST /identity/v3/services => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 06:00:43.126155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303741a-6b1f-436e-8696-1d8d565110f0 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:43.173775 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc8c1-f20e-4e37-8c51-154277474823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:43.174728 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc8c1-f20e-4e37-8c51-154277474823 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:43.175156 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc8c1-f20e-4e37-8c51-154277474823 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:43.175575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc8c1-f20e-4e37-8c51-154277474823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:43.188030 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdc8c1-f20e-4e37-8c51-154277474823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:43.190284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdc8c1-f20e-4e37-8c51-154277474823 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-39fdc8c1-f20e-4e37-8c51-154277474823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:43.209412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:43.210658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:43.211733 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:43.211733 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:43.211733 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:43.213605 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 432/869] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:43.219449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378443ab-fd05-4b96-9506-de6f108f8b76 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:43.282730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:43.338332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378443ab-fd05-4b96-9506-de6f108f8b76 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:43.374064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e04129-599b-4a96-9919-b4ec63d03f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:43.374809 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e04129-599b-4a96-9919-b4ec63d03f46 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:43.375084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e04129-599b-4a96-9919-b4ec63d03f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:43.375351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e04129-599b-4a96-9919-b4ec63d03f46 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:43.375992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e04129-599b-4a96-9919-b4ec63d03f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:43.376294 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e04129-599b-4a96-9919-b4ec63d03f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:43.377659 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e04129-599b-4a96-9919-b4ec63d03f46 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:43.391050 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 438/870] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:43.396381 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc8c1-f20e-4e37-8c51-154277474823 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:43.499607 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:43.503638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc8c1-f20e-4e37-8c51-154277474823 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:43.542727 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73033f4-09ee-4d89-9e46-87669a02a7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:43.543491 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73033f4-09ee-4d89-9e46-87669a02a7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:43.543771 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73033f4-09ee-4d89-9e46-87669a02a7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:43.544034 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73033f4-09ee-4d89-9e46-87669a02a7c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:43.551486 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73033f4-09ee-4d89-9e46-87669a02a7c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:43.552948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73033f4-09ee-4d89-9e46-87669a02a7c5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:43.559659 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07055765-9217-41c6-b954-33fc035a0fb3 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:43.559980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07055765-9217-41c6-b954-33fc035a0fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:43.560346 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07055765-9217-41c6-b954-33fc035a0fb3 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:43.561899 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 439/871] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:43.565726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-c73033f4-09ee-4d89-9e46-87669a02a7c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:43.565726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:43.565726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:43.565726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:43.565726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:43.565726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:43.565726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:43.565726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:43.565726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:43.565726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:43.565726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:43.565726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:43.565726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:43.565726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:43.565726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:43.566628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:43.567889 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:43.567889 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:43.567889 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:43.567889 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:43.567889 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:43.567889 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:43.567889 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:43.567889 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:43.567889 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 433/872] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:43.568586 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a6ebb-58dd-43c0-8e04-f9306428841a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:43.568586 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a6ebb-58dd-43c0-8e04-f9306428841a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:43.568586 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a6ebb-58dd-43c0-8e04-f9306428841a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:43.572661 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73033f4-09ee-4d89-9e46-87669a02a7c5 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:43.591902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-f01a6ebb-58dd-43c0-8e04-f9306428841a None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:43.593728 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f01a6ebb-58dd-43c0-8e04-f9306428841a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:43.681950 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 440/873] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:43.690731 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73033f4-09ee-4d89-9e46-87669a02a7c5 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:43.738508 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b66db-b301-4259-a1c2-1e442276ad7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:43.739149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b66db-b301-4259-a1c2-1e442276ad7e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:43.739429 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b66db-b301-4259-a1c2-1e442276ad7e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:43.739769 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b66db-b301-4259-a1c2-1e442276ad7e None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:43.740433 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b66db-b301-4259-a1c2-1e442276ad7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:43.740755 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b66db-b301-4259-a1c2-1e442276ad7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:43.742263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b66db-b301-4259-a1c2-1e442276ad7e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:43.757514 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 434/874] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:43.763780 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a6ebb-58dd-43c0-8e04-f9306428841a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:43.809498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:43.871057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a6ebb-58dd-43c0-8e04-f9306428841a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:43.909781 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a2def-8bdd-4291-916e-847afda02c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:43.910392 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a2def-8bdd-4291-916e-847afda02c88 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:43.910695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a2def-8bdd-4291-916e-847afda02c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:43.910967 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a2def-8bdd-4291-916e-847afda02c88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:43.911704 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a2def-8bdd-4291-916e-847afda02c88 None admin] RBAC: Adding query filter params (role.id=3209a3d044b84a0ebd7cf1edf2da7120, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=078a012fea464954b0632a6ce86d7cec) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:43.912008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a2def-8bdd-4291-916e-847afda02c88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:43.913434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a2def-8bdd-4291-916e-847afda02c88 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:43.921301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 441/875] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:43 2022] GET /identity/v3/role_assignments?user.id=078a012fea464954b0632a6ce86d7cec&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=3209a3d044b84a0ebd7cf1edf2da7120 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:44.011398 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b66db-b301-4259-a1c2-1e442276ad7e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:44.029872 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:44.154724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b66db-b301-4259-a1c2-1e442276ad7e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:44.196229 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0068a6d-4d79-4c70-b967-d1a8a46204d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:44.197559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0068a6d-4d79-4c70-b967-d1a8a46204d7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:44.197845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0068a6d-4d79-4c70-b967-d1a8a46204d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:44.198146 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0068a6d-4d79-4c70-b967-d1a8a46204d7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:44.198778 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0068a6d-4d79-4c70-b967-d1a8a46204d7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:44.201006 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0068a6d-4d79-4c70-b967-d1a8a46204d7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-f0068a6d-4d79-4c70-b967-d1a8a46204d7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:44.210303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:44.211416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:44.212239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:44.212239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:44.212239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:44.212239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:44.212239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:44.212239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:44.212239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:44.212239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:44.212239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:44.212239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:44.212239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:44.212239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:44.212239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 06:00:44.212239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:44.214629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 435/876] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 23 06:00:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:44.218311 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a2def-8bdd-4291-916e-847afda02c88 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:44.333798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a2def-8bdd-4291-916e-847afda02c88 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:44.336955 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:44.372368 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34085af5-9947-486c-8a6d-594e8c2e2c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:44.372980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34085af5-9947-486c-8a6d-594e8c2e2c1d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:44.373258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34085af5-9947-486c-8a6d-594e8c2e2c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:44.373528 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34085af5-9947-486c-8a6d-594e8c2e2c1d None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:44.374159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34085af5-9947-486c-8a6d-594e8c2e2c1d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:44.374459 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34085af5-9947-486c-8a6d-594e8c2e2c1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:44.375857 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34085af5-9947-486c-8a6d-594e8c2e2c1d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:44.381097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 442/877] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 23 06:00:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:44.386677 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0068a6d-4d79-4c70-b967-d1a8a46204d7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:44.393609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41db427-53f8-4c09-ba64-e4d590ad364b None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:44.393896 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41db427-53f8-4c09-ba64-e4d590ad364b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:44.394174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41db427-53f8-4c09-ba64-e4d590ad364b None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:44.395815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 443/878] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:00:44.400875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e0ee8-2790-46db-9000-77d539fe8557 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:44.401268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e0ee8-2790-46db-9000-77d539fe8557 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:44.401548 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e0ee8-2790-46db-9000-77d539fe8557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:44.423125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-241e0ee8-2790-46db-9000-77d539fe8557 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:44.424354 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-241e0ee8-2790-46db-9000-77d539fe8557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:44.490291 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:44.511487 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 444/879] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:44 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:44.516594 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0068a6d-4d79-4c70-b967-d1a8a46204d7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:44.553356 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de117b-8875-447b-9b5b-2e11e290fbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:44.553961 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de117b-8875-447b-9b5b-2e11e290fbee None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:44.554241 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de117b-8875-447b-9b5b-2e11e290fbee None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:44.554531 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de117b-8875-447b-9b5b-2e11e290fbee None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:44.555159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de117b-8875-447b-9b5b-2e11e290fbee None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:44.555461 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de117b-8875-447b-9b5b-2e11e290fbee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:44.556827 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de117b-8875-447b-9b5b-2e11e290fbee None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:44.561886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 436/880] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 23 06:00:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:44.787057 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e0ee8-2790-46db-9000-77d539fe8557 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:44.896863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:44.901258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e0ee8-2790-46db-9000-77d539fe8557 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:44.938214 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d257c-6eda-4c8a-bc3e-abdd83956083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:44.938856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d257c-6eda-4c8a-bc3e-abdd83956083 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:44.939133 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d257c-6eda-4c8a-bc3e-abdd83956083 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:44.939401 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d257c-6eda-4c8a-bc3e-abdd83956083 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:44.940060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d257c-6eda-4c8a-bc3e-abdd83956083 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:44.941428 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d257c-6eda-4c8a-bc3e-abdd83956083 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:44.968954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 445/881] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 23 06:00:44 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:00:45.247892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae59f96-5b08-46b6-8879-6c2f712611ce None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:45.248384 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae59f96-5b08-46b6-8879-6c2f712611ce None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:45.248384 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae59f96-5b08-46b6-8879-6c2f712611ce None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:45.249910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 437/882] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:45.256819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d5716-72c6-4562-8cd3-c881709a430a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:45.257105 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d5716-72c6-4562-8cd3-c881709a430a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:45.257379 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d5716-72c6-4562-8cd3-c881709a430a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:45.275600 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-f14d5716-72c6-4562-8cd3-c881709a430a None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:45.276793 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f14d5716-72c6-4562-8cd3-c881709a430a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:45.351150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 446/883] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:45.617435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae59f96-5b08-46b6-8879-6c2f712611ce None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:45.728827 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae59f96-5b08-46b6-8879-6c2f712611ce None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:45.768073 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc12481-074e-4f78-81d4-1d833174db1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:45.768703 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc12481-074e-4f78-81d4-1d833174db1e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:45.768978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc12481-074e-4f78-81d4-1d833174db1e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:45.769247 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc12481-074e-4f78-81d4-1d833174db1e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:45.769826 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc12481-074e-4f78-81d4-1d833174db1e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:45.771316 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc12481-074e-4f78-81d4-1d833174db1e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-fbc12481-074e-4f78-81d4-1d833174db1e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:45.780932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:45.781925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:45.782817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:45.782817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:45.782817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:45.782817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:45.782817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:45.782817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:45.782817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:45.782817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:45.782817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:45.782817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:45.782817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:45.782817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:45.782817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 06:00:45.782817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:45.782817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 438/884] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 06:00:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:45.788257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d5716-72c6-4562-8cd3-c881709a430a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:45.880104 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:45.899736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d5716-72c6-4562-8cd3-c881709a430a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:45.938330 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2616a9-f38d-4f44-b994-37323c134d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:45.939333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2616a9-f38d-4f44-b994-37323c134d1a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:45.939816 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2616a9-f38d-4f44-b994-37323c134d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:45.940285 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2616a9-f38d-4f44-b994-37323c134d1a None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:45.941159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2616a9-f38d-4f44-b994-37323c134d1a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:45.941674 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2616a9-f38d-4f44-b994-37323c134d1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:45.943613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2616a9-f38d-4f44-b994-37323c134d1a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:45.949398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 447/885] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 06:00:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:45.958730 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc12481-074e-4f78-81d4-1d833174db1e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:45.974921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:46.069680 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc12481-074e-4f78-81d4-1d833174db1e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:46.074281 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7246e8d-eb45-4f38-b6b1-d261e12dca7b None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:46.074745 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7246e8d-eb45-4f38-b6b1-d261e12dca7b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:46.075234 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7246e8d-eb45-4f38-b6b1-d261e12dca7b None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:46.076912 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 448/886] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:00:46.083868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21c245-0c16-4310-9291-de365c9b172d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:46.084278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21c245-0c16-4310-9291-de365c9b172d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:46.084654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21c245-0c16-4310-9291-de365c9b172d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:46.104076 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1d21c245-0c16-4310-9291-de365c9b172d None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:46.105595 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1d21c245-0c16-4310-9291-de365c9b172d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:46.110860 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81e722-d66f-4ccb-b851-fb392bb729a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:46.111673 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81e722-d66f-4ccb-b851-fb392bb729a8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:46.112117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81e722-d66f-4ccb-b851-fb392bb729a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:46.112527 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81e722-d66f-4ccb-b851-fb392bb729a8 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:46.113312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81e722-d66f-4ccb-b851-fb392bb729a8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:46.113806 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81e722-d66f-4ccb-b851-fb392bb729a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:46.115518 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81e722-d66f-4ccb-b851-fb392bb729a8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:46.122146 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 439/887] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 06:00:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:46.128311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81e722-d66f-4ccb-b851-fb392bb729a8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:46.190750 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 449/888] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:00:46.203926 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:46.240921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81e722-d66f-4ccb-b851-fb392bb729a8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:46.284789 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebaf9a5-d5d8-4465-b870-5c9e1cb17083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:46.286174 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebaf9a5-d5d8-4465-b870-5c9e1cb17083 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:46.286642 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebaf9a5-d5d8-4465-b870-5c9e1cb17083 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:46.287063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebaf9a5-d5d8-4465-b870-5c9e1cb17083 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:46.287904 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebaf9a5-d5d8-4465-b870-5c9e1cb17083 None admin] RBAC: Adding query filter params (interface=public, service_id=693f5564428c4c089aaa86952bf3d759, region_id=RegionOne) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:46.288392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebaf9a5-d5d8-4465-b870-5c9e1cb17083 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:46.289921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebaf9a5-d5d8-4465-b870-5c9e1cb17083 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:46.295755 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 440/889] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 23 06:00:46 2022] GET /identity/v3/endpoints?service_id=693f5564428c4c089aaa86952bf3d759&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:46.301561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21c245-0c16-4310-9291-de365c9b172d None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:46.389965 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:46.412801 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21c245-0c16-4310-9291-de365c9b172d None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:46.450349 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bded57b-1205-4965-abb2-301e146a3146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:46.451118 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bded57b-1205-4965-abb2-301e146a3146 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:46.451403 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bded57b-1205-4965-abb2-301e146a3146 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:46.451677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bded57b-1205-4965-abb2-301e146a3146 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:46.452286 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bded57b-1205-4965-abb2-301e146a3146 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:46.453658 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bded57b-1205-4965-abb2-301e146a3146 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:46.460026 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 450/890] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 23 06:00:46 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 06:00:46.493990 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebaf9a5-d5d8-4465-b870-5c9e1cb17083 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:46.563344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a34aca-0207-4efc-a753-986b3f77d70f None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:46.563690 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a34aca-0207-4efc-a753-986b3f77d70f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:46.563914 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a34aca-0207-4efc-a753-986b3f77d70f None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:46.565632 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 451/891] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:00:46.571284 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db485b-7dff-4110-95bd-4fd57d64c403 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:46.571602 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db485b-7dff-4110-95bd-4fd57d64c403 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:46.571831 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db485b-7dff-4110-95bd-4fd57d64c403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:46.591995 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-88db485b-7dff-4110-95bd-4fd57d64c403 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:46.593230 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-88db485b-7dff-4110-95bd-4fd57d64c403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:46.624670 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebaf9a5-d5d8-4465-b870-5c9e1cb17083 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:46.665894 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bea45-39df-4581-b170-c7579155cd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:46.666587 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bea45-39df-4581-b170-c7579155cd84 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:46.668470 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bea45-39df-4581-b170-c7579155cd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:46.668751 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bea45-39df-4581-b170-c7579155cd84 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:46.671513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:46.684541 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5bea45-39df-4581-b170-c7579155cd84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-3f5bea45-39df-4581-b170-c7579155cd84 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:46.691194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:46.692226 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:46.693536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:46.693536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:46.693536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:46.693536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:46.693536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:46.693536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:46.693536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:46.693536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:46.693536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:46.693536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:46.693536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 06:00:46.693536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:46.693536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 441/892] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 23 06:00:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:46.697335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 452/893] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:00:46.700282 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bea45-39df-4581-b170-c7579155cd84 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:46.809962 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bea45-39df-4581-b170-c7579155cd84 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:46.846736 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365e83f-624f-4b4b-9a43-30eff9431568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:46.847668 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365e83f-624f-4b4b-9a43-30eff9431568 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:46.848148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365e83f-624f-4b4b-9a43-30eff9431568 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:46.848617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365e83f-624f-4b4b-9a43-30eff9431568 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:46.849466 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365e83f-624f-4b4b-9a43-30eff9431568 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:46.849959 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365e83f-624f-4b4b-9a43-30eff9431568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:46.851536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365e83f-624f-4b4b-9a43-30eff9431568 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:46.860749 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 442/894] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 23 06:00:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:46.867650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db485b-7dff-4110-95bd-4fd57d64c403 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:46.952123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:46.983952 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db485b-7dff-4110-95bd-4fd57d64c403 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:46.997656 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365e83f-624f-4b4b-9a43-30eff9431568 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:47.025431 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1898c0-f2cf-41cf-a4c8-1e2d521a440d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:47.026073 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1898c0-f2cf-41cf-a4c8-1e2d521a440d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:47.026352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1898c0-f2cf-41cf-a4c8-1e2d521a440d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:47.026654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1898c0-f2cf-41cf-a4c8-1e2d521a440d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:47.039036 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1898c0-f2cf-41cf-a4c8-1e2d521a440d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:47.040486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1898c0-f2cf-41cf-a4c8-1e2d521a440d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-cf1898c0-f2cf-41cf-a4c8-1e2d521a440d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:47.057918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:47.059157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:47.060099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:47.060099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:47.060099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:47.064360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 453/895] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:47.065428 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1898c0-f2cf-41cf-a4c8-1e2d521a440d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:47.069167 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:47.116742 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365e83f-624f-4b4b-9a43-30eff9431568 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:47.154558 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2840a-0ac9-40a8-a306-516491eeaa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:47.155281 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2840a-0ac9-40a8-a306-516491eeaa72 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:47.155570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2840a-0ac9-40a8-a306-516491eeaa72 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:47.155843 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2840a-0ac9-40a8-a306-516491eeaa72 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:47.156504 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2840a-0ac9-40a8-a306-516491eeaa72 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:47.157875 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2840a-0ac9-40a8-a306-516491eeaa72 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:47.177103 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 443/896] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 23 06:00:46 2022] POST /identity/v3/services => generated 241 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:00:47.178287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c74174-f162-4c38-998c-c9058f105f4d None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:47.178606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c74174-f162-4c38-998c-c9058f105f4d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:47.178885 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c74174-f162-4c38-998c-c9058f105f4d None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:47.180259 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 444/897] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:47.186726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f015df-f0f6-4a93-900d-f13d56c42757 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:47.187062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f015df-f0f6-4a93-900d-f13d56c42757 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:47.187284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f015df-f0f6-4a93-900d-f13d56c42757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:47.197400 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1898c0-f2cf-41cf-a4c8-1e2d521a440d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:47.208350 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-73f015df-f0f6-4a93-900d-f13d56c42757 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:47.210603 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-73f015df-f0f6-4a93-900d-f13d56c42757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:47.232063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:47.257900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9457e53-4633-4782-ac10-3f942f67bca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:47.258977 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9457e53-4633-4782-ac10-3f942f67bca9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:47.259467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9457e53-4633-4782-ac10-3f942f67bca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:47.259950 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9457e53-4633-4782-ac10-3f942f67bca9 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:47.260872 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9457e53-4633-4782-ac10-3f942f67bca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:47.262145 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9457e53-4633-4782-ac10-3f942f67bca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:47.263998 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9457e53-4633-4782-ac10-3f942f67bca9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:47.283635 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 454/898] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:47.288904 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9457e53-4633-4782-ac10-3f942f67bca9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:47.300741 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 445/899] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:00:47.406628 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9457e53-4633-4782-ac10-3f942f67bca9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:47.444034 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b215a1-b478-4a50-a789-98fe58c2d044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:47.444681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b215a1-b478-4a50-a789-98fe58c2d044 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:47.445054 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b215a1-b478-4a50-a789-98fe58c2d044 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:47.445410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b215a1-b478-4a50-a789-98fe58c2d044 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:47.453695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b215a1-b478-4a50-a789-98fe58c2d044 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:47.455179 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b215a1-b478-4a50-a789-98fe58c2d044 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-27b215a1-b478-4a50-a789-98fe58c2d044 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:47.468575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:47.469563 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:47.471123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:47.471123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:47.471123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:47.471123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:47.471123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:47.471123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:47.471123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:47.471123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:47.471123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 06:00:47.471123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:47.471123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 455/900] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 23 06:00:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:47.475738 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f015df-f0f6-4a93-900d-f13d56c42757 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:47.528278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:47.589052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f015df-f0f6-4a93-900d-f13d56c42757 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:47.606418 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b215a1-b478-4a50-a789-98fe58c2d044 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:47.638194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace458f8-31b1-4717-bb3e-6caacc25e908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:47.638849 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace458f8-31b1-4717-bb3e-6caacc25e908 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:47.639189 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace458f8-31b1-4717-bb3e-6caacc25e908 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:47.639464 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace458f8-31b1-4717-bb3e-6caacc25e908 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:47.640528 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace458f8-31b1-4717-bb3e-6caacc25e908 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:47.640841 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace458f8-31b1-4717-bb3e-6caacc25e908 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:47.642978 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace458f8-31b1-4717-bb3e-6caacc25e908 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:47.655250 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:47.665904 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 446/901] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 23 06:00:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:47.672422 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace458f8-31b1-4717-bb3e-6caacc25e908 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:47.729427 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b215a1-b478-4a50-a789-98fe58c2d044 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:47.767953 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807c032-b2d6-4c39-b010-8d2c0658b8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:47.768982 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807c032-b2d6-4c39-b010-8d2c0658b8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:47.769457 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807c032-b2d6-4c39-b010-8d2c0658b8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:47.769910 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807c032-b2d6-4c39-b010-8d2c0658b8d2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:47.770729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807c032-b2d6-4c39-b010-8d2c0658b8d2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:47.772474 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807c032-b2d6-4c39-b010-8d2c0658b8d2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:47.781184 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace458f8-31b1-4717-bb3e-6caacc25e908 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:47.783524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-8807c032-b2d6-4c39-b010-8d2c0658b8d2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:47.784915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:47.786453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:47.787410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:47.787410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:47.787410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:47.787410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:47.787410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:47.787410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:47.787410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:47.787410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:47.787410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:47.787410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:47.787410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:47.787410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:47.787410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 06:00:47.787410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:47.790001 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 456/902] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Aug 23 06:00:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:47.799776 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807c032-b2d6-4c39-b010-8d2c0658b8d2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:47.826422 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d552c-0948-444e-9c87-367fba6821e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:47.827469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d552c-0948-444e-9c87-367fba6821e4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:47.827755 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d552c-0948-444e-9c87-367fba6821e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:47.828028 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d552c-0948-444e-9c87-367fba6821e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:47.915533 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d552c-0948-444e-9c87-367fba6821e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:47.917086 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d552c-0948-444e-9c87-367fba6821e4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:47.952934 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807c032-b2d6-4c39-b010-8d2c0658b8d2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:47.960991 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:47.995757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b935b36-1555-4bb8-9108-753e9032bbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:47.996960 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b935b36-1555-4bb8-9108-753e9032bbfb None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:47.997242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b935b36-1555-4bb8-9108-753e9032bbfb None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:47.997595 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b935b36-1555-4bb8-9108-753e9032bbfb None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:47.998220 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b935b36-1555-4bb8-9108-753e9032bbfb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:47.998529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b935b36-1555-4bb8-9108-753e9032bbfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:48.000233 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b935b36-1555-4bb8-9108-753e9032bbfb None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:48.005639 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 457/903] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 23 06:00:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:48.018599 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b935b36-1555-4bb8-9108-753e9032bbfb None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:48.059316 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 447/904] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:47 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:00:48.060146 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d552c-0948-444e-9c87-367fba6821e4 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:48.064825 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:48.158717 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b935b36-1555-4bb8-9108-753e9032bbfb None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:48.184205 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d552c-0948-444e-9c87-367fba6821e4 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:48.196583 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d046e93-e596-4009-9f26-7bf58c765f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:48.197602 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d046e93-e596-4009-9f26-7bf58c765f6f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:48.198094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d046e93-e596-4009-9f26-7bf58c765f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:48.198635 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d046e93-e596-4009-9f26-7bf58c765f6f None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:48.200160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d046e93-e596-4009-9f26-7bf58c765f6f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:48.200685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d046e93-e596-4009-9f26-7bf58c765f6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:48.202275 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d046e93-e596-4009-9f26-7bf58c765f6f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:48.208244 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 458/905] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 23 06:00:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:48.216820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d046e93-e596-4009-9f26-7bf58c765f6f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:48.227090 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb7742-a8ff-4331-8307-c6a7d35fe06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:48.227569 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb7742-a8ff-4331-8307-c6a7d35fe06a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:48.227850 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb7742-a8ff-4331-8307-c6a7d35fe06a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:48.228127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb7742-a8ff-4331-8307-c6a7d35fe06a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:48.239593 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb7742-a8ff-4331-8307-c6a7d35fe06a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:48.240931 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb7742-a8ff-4331-8307-c6a7d35fe06a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:48.255501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-5eeb7742-a8ff-4331-8307-c6a7d35fe06a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:48.259846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:48.260791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:48.262326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:48.262326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:48.262326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:48.266403 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 448/906] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:48.267696 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb7742-a8ff-4331-8307-c6a7d35fe06a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:48.331129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:48.341589 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d046e93-e596-4009-9f26-7bf58c765f6f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:48.379580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f0101-486b-44a3-9602-5ec6091b3f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:48.380541 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f0101-486b-44a3-9602-5ec6091b3f21 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:48.381024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f0101-486b-44a3-9602-5ec6091b3f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:48.381529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f0101-486b-44a3-9602-5ec6091b3f21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:48.382149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb7742-a8ff-4331-8307-c6a7d35fe06a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:48.382705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f0101-486b-44a3-9602-5ec6091b3f21 None admin] RBAC: Adding query filter params (interface=public, service_id=4d7abc65388748daa6d7db2207d8ff0c, region_id=RegionOne) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:48.383218 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f0101-486b-44a3-9602-5ec6091b3f21 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:48.384787 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f0101-486b-44a3-9602-5ec6091b3f21 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:48.394628 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 459/907] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 23 06:00:48 2022] GET /identity/v3/endpoints?service_id=4d7abc65388748daa6d7db2207d8ff0c&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:48.397383 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f0101-486b-44a3-9602-5ec6091b3f21 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:48.422120 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea01c6f-ec95-444f-aa82-6c3f7d35de1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:48.422821 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01c6f-ec95-444f-aa82-6c3f7d35de1d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:48.423159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01c6f-ec95-444f-aa82-6c3f7d35de1d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:48.423421 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01c6f-ec95-444f-aa82-6c3f7d35de1d None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:48.424063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea01c6f-ec95-444f-aa82-6c3f7d35de1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:48.424404 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea01c6f-ec95-444f-aa82-6c3f7d35de1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:48.425923 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea01c6f-ec95-444f-aa82-6c3f7d35de1d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:48.438696 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 449/908] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:48.444327 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc1fa1-cb36-4dda-b83c-d4063f31ee8e None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:48.444614 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc1fa1-cb36-4dda-b83c-d4063f31ee8e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:48.444915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc1fa1-cb36-4dda-b83c-d4063f31ee8e None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:48.446332 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 450/909] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:48.447279 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc1fa1-cb36-4dda-b83c-d4063f31ee8e None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:48.508857 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:48.516906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f0101-486b-44a3-9602-5ec6091b3f21 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:48.555519 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a67b9-4481-4de6-86c1-dbfc68530d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:48.556128 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a67b9-4481-4de6-86c1-dbfc68530d0f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:48.556406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a67b9-4481-4de6-86c1-dbfc68530d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:48.556815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a67b9-4481-4de6-86c1-dbfc68530d0f None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:48.557874 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a67b9-4481-4de6-86c1-dbfc68530d0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:48.557977 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc1fa1-cb36-4dda-b83c-d4063f31ee8e None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:48.559903 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a67b9-4481-4de6-86c1-dbfc68530d0f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:48.574788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 460/910] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 23 06:00:48 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 06:00:48.574788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d545905-dc66-4b39-b6ba-ba64fe26babe None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:48.574788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d545905-dc66-4b39-b6ba-ba64fe26babe None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:48.574788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d545905-dc66-4b39-b6ba-ba64fe26babe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:48.583204 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:48.604954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-3d545905-dc66-4b39-b6ba-ba64fe26babe None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:48.606435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3d545905-dc66-4b39-b6ba-ba64fe26babe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:48.611708 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece2a2a-66e6-4d36-8e00-232a1a008d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:48.612362 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece2a2a-66e6-4d36-8e00-232a1a008d9f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:48.612641 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece2a2a-66e6-4d36-8e00-232a1a008d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:48.612912 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece2a2a-66e6-4d36-8e00-232a1a008d9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:48.623246 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece2a2a-66e6-4d36-8e00-232a1a008d9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:48.625597 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece2a2a-66e6-4d36-8e00-232a1a008d9f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:48.642015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-fece2a2a-66e6-4d36-8e00-232a1a008d9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:48.642015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:48.642015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:48.642015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:48.642015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:48.642015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:48.642015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:48.642015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:48.642015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:48.642015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:48.642015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:48.642015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:48.642015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:48.642015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:48.642015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:48.644055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:48.646198 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:48.646198 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:48.646198 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:48.646198 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:48.646198 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:48.646198 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:48.646198 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:48.646198 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:48.650985 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 451/911] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:48.650985 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece2a2a-66e6-4d36-8e00-232a1a008d9f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:48.739577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 461/912] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:48.781422 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece2a2a-66e6-4d36-8e00-232a1a008d9f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:48.818795 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dc898-4df8-47d3-a105-c075a29cc728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:48.819352 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dc898-4df8-47d3-a105-c075a29cc728 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:48.819626 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dc898-4df8-47d3-a105-c075a29cc728 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:48.819895 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dc898-4df8-47d3-a105-c075a29cc728 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:48.820604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dc898-4df8-47d3-a105-c075a29cc728 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:48.820909 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dc898-4df8-47d3-a105-c075a29cc728 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:48.822361 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dc898-4df8-47d3-a105-c075a29cc728 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:48.841066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 452/913] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:48.843334 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d545905-dc66-4b39-b6ba-ba64fe26babe None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:48.945542 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:48.954683 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d545905-dc66-4b39-b6ba-ba64fe26babe None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:48.990628 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f70d04-08a8-4e35-910f-60b4417472ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:48.991177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f70d04-08a8-4e35-910f-60b4417472ea None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:48.991456 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f70d04-08a8-4e35-910f-60b4417472ea None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:48.991730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f70d04-08a8-4e35-910f-60b4417472ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:48.992456 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f70d04-08a8-4e35-910f-60b4417472ea None admin] RBAC: Adding query filter params (role.id=aa12d72368054e13a59fa2a814d51afd, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=078a012fea464954b0632a6ce86d7cec) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:48.992758 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f70d04-08a8-4e35-910f-60b4417472ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:48.994583 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f70d04-08a8-4e35-910f-60b4417472ea None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:49.005740 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 462/914] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:48 2022] GET /identity/v3/role_assignments?user.id=078a012fea464954b0632a6ce86d7cec&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=aa12d72368054e13a59fa2a814d51afd => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:49.019946 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dc898-4df8-47d3-a105-c075a29cc728 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:49.062272 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:49.140435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dc898-4df8-47d3-a105-c075a29cc728 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:49.180389 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42e494-55dd-43c8-afbd-4205318b8a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:49.181020 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42e494-55dd-43c8-afbd-4205318b8a4a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:49.181296 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42e494-55dd-43c8-afbd-4205318b8a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:49.181569 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42e494-55dd-43c8-afbd-4205318b8a4a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:49.182150 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42e494-55dd-43c8-afbd-4205318b8a4a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:49.183580 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42e494-55dd-43c8-afbd-4205318b8a4a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-eb42e494-55dd-43c8-afbd-4205318b8a4a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:49.192629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:49.193679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:49.194553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:49.194553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:49.194553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:49.194553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:49.194553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:49.194553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:49.194553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:49.194553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:49.194553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:49.194553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:49.194553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:49.194553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:49.194553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 06:00:49.194553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:49.194553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 453/915] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 06:00:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:49.203636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f70d04-08a8-4e35-910f-60b4417472ea None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:49.241238 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23df360-7caa-4d29-8bfe-7b94da738f38 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:49.241484 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23df360-7caa-4d29-8bfe-7b94da738f38 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:49.241731 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23df360-7caa-4d29-8bfe-7b94da738f38 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:49.245786 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 454/916] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:49.251800 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b0192-aa32-4c29-aac6-1b6491d6ece8 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:49.252071 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b0192-aa32-4c29-aac6-1b6491d6ece8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:49.252348 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b0192-aa32-4c29-aac6-1b6491d6ece8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:49.289680 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-be3b0192-aa32-4c29-aac6-1b6491d6ece8 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:49.290905 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-be3b0192-aa32-4c29-aac6-1b6491d6ece8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:49.369547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f70d04-08a8-4e35-910f-60b4417472ea None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:49.417428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 455/917] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:00:49.463846 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319d2ae-d2c5-4629-b93b-37863e79e0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:49.464468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319d2ae-d2c5-4629-b93b-37863e79e0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:49.465341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319d2ae-d2c5-4629-b93b-37863e79e0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:49.465615 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319d2ae-d2c5-4629-b93b-37863e79e0d3 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:49.466253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319d2ae-d2c5-4629-b93b-37863e79e0d3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:49.466568 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319d2ae-d2c5-4629-b93b-37863e79e0d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:49.467936 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319d2ae-d2c5-4629-b93b-37863e79e0d3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:49.484803 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 463/918] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 06:00:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:49.490610 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b0192-aa32-4c29-aac6-1b6491d6ece8 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:49.575225 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:49.660029 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b0192-aa32-4c29-aac6-1b6491d6ece8 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:49.737981 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319d2ae-d2c5-4629-b93b-37863e79e0d3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:49.759942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:49.760611 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec625bd-77b3-4011-955a-fec1dfd82dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:49.761376 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec625bd-77b3-4011-955a-fec1dfd82dfa None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:49.761752 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec625bd-77b3-4011-955a-fec1dfd82dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:49.762155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec625bd-77b3-4011-955a-fec1dfd82dfa None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:49.762832 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec625bd-77b3-4011-955a-fec1dfd82dfa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:49.763133 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec625bd-77b3-4011-955a-fec1dfd82dfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:49.764896 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec625bd-77b3-4011-955a-fec1dfd82dfa None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:49.773621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 456/919] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Aug 23 06:00:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:49.784225 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec625bd-77b3-4011-955a-fec1dfd82dfa None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:49.872646 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319d2ae-d2c5-4629-b93b-37863e79e0d3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:49.897306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:49.900641 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec625bd-77b3-4011-955a-fec1dfd82dfa None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:49.912134 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7ec88-665f-4636-8ec6-6ad1041633d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:49.912771 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7ec88-665f-4636-8ec6-6ad1041633d7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:49.913077 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7ec88-665f-4636-8ec6-6ad1041633d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:49.913347 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7ec88-665f-4636-8ec6-6ad1041633d7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:49.913932 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7ec88-665f-4636-8ec6-6ad1041633d7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:49.915336 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7ec88-665f-4636-8ec6-6ad1041633d7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-20f7ec88-665f-4636-8ec6-6ad1041633d7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:49.924707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:49.925748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:49.927046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:49.927046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:49.927046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:49.927046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:49.927046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:49.927046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:49.927046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:49.927046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:49.927046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:49.927046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:49.927046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:49.927046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:49.927046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 06:00:49.927046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:49.930787 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 464/920] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 23 06:00:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:49.931657 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7ec88-665f-4636-8ec6-6ad1041633d7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:49.951274 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1afb7-5b3d-4272-aa52-ebddfa8e078a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:49.952769 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1afb7-5b3d-4272-aa52-ebddfa8e078a None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:49.953259 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1afb7-5b3d-4272-aa52-ebddfa8e078a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:49.953584 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1afb7-5b3d-4272-aa52-ebddfa8e078a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:49.954246 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1afb7-5b3d-4272-aa52-ebddfa8e078a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:49.957953 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1afb7-5b3d-4272-aa52-ebddfa8e078a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:49.985093 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 457/921] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 23 06:00:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 06:00:50.050114 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:50.073119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7ec88-665f-4636-8ec6-6ad1041633d7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:50.122217 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de354e-33fa-4118-a230-0e105013f2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:50.123329 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de354e-33fa-4118-a230-0e105013f2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:50.123812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de354e-33fa-4118-a230-0e105013f2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:50.124285 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de354e-33fa-4118-a230-0e105013f2b2 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:50.125197 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de354e-33fa-4118-a230-0e105013f2b2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:50.125719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de354e-33fa-4118-a230-0e105013f2b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:50.127449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de354e-33fa-4118-a230-0e105013f2b2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:50.134090 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 465/922] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 23 06:00:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:50.141331 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1afb7-5b3d-4272-aa52-ebddfa8e078a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:50.251149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1afb7-5b3d-4272-aa52-ebddfa8e078a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:50.291080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d8264-6896-4735-8bd4-b69fe20e060e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:50.292071 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d8264-6896-4735-8bd4-b69fe20e060e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:50.292548 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d8264-6896-4735-8bd4-b69fe20e060e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:50.293053 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d8264-6896-4735-8bd4-b69fe20e060e None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:50.294085 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d8264-6896-4735-8bd4-b69fe20e060e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:50.294645 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d8264-6896-4735-8bd4-b69fe20e060e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:50.296314 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d8264-6896-4735-8bd4-b69fe20e060e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:50.302094 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 458/923] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 23 06:00:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:50.308170 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de354e-33fa-4118-a230-0e105013f2b2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:50.417071 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de354e-33fa-4118-a230-0e105013f2b2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:50.454470 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f07c2-adcf-4452-ba18-5af59d704035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:50.455488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f07c2-adcf-4452-ba18-5af59d704035 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:50.455968 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f07c2-adcf-4452-ba18-5af59d704035 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:50.456446 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f07c2-adcf-4452-ba18-5af59d704035 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:50.457365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f07c2-adcf-4452-ba18-5af59d704035 None admin] RBAC: Adding query filter params (interface=public, service_id=d6dac826bf7b4966a3d7f9cd97fd8598, region_id=RegionOne) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:50.457877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f07c2-adcf-4452-ba18-5af59d704035 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:50.459461 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f07c2-adcf-4452-ba18-5af59d704035 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:50.465932 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 466/924] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 23 06:00:50 2022] GET /identity/v3/endpoints?service_id=d6dac826bf7b4966a3d7f9cd97fd8598&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:50.472499 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d8264-6896-4735-8bd4-b69fe20e060e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:50.491528 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:50.584178 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d8264-6896-4735-8bd4-b69fe20e060e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:50.624902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6367b4-0e81-4151-86a5-4689321a6e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:50.625522 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6367b4-0e81-4151-86a5-4689321a6e16 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:50.625801 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6367b4-0e81-4151-86a5-4689321a6e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:50.626074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6367b4-0e81-4151-86a5-4689321a6e16 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:50.626783 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6367b4-0e81-4151-86a5-4689321a6e16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:50.628172 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6367b4-0e81-4151-86a5-4689321a6e16 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:50.635003 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 459/925] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 23 06:00:50 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 06:00:50.756748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d71df5-ffcf-4097-8b3b-bfae14926d60 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:50.757053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d71df5-ffcf-4097-8b3b-bfae14926d60 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:50.757296 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d71df5-ffcf-4097-8b3b-bfae14926d60 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:50.759092 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 467/926] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:50.765657 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f9e64-5e19-49a3-8b2b-5b3b2df7f3de None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:50.765977 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f9e64-5e19-49a3-8b2b-5b3b2df7f3de None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:50.766203 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f9e64-5e19-49a3-8b2b-5b3b2df7f3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:50.785809 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-f55f9e64-5e19-49a3-8b2b-5b3b2df7f3de None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:50.787359 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f55f9e64-5e19-49a3-8b2b-5b3b2df7f3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:50.866860 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 460/927] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:50.995751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5640e-62b0-4104-8b29-91a93a881551 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:50.996374 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5640e-62b0-4104-8b29-91a93a881551 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:50.996865 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5640e-62b0-4104-8b29-91a93a881551 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:50.998087 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 468/928] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:51.003394 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a109e-74ea-4274-bed1-d52cc323bca1 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:51.003676 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a109e-74ea-4274-bed1-d52cc323bca1 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:51.003948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a109e-74ea-4274-bed1-d52cc323bca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:51.024854 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-402a109e-74ea-4274-bed1-d52cc323bca1 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:51.026342 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-402a109e-74ea-4274-bed1-d52cc323bca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:51.029610 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:51.101238 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 461/929] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:51.158023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5640e-62b0-4104-8b29-91a93a881551 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:51.169371 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:51.276260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5640e-62b0-4104-8b29-91a93a881551 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:51.316172 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de5ba1-ffe6-4d04-ad7b-01429e616d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:51.317191 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de5ba1-ffe6-4d04-ad7b-01429e616d66 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:51.317670 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de5ba1-ffe6-4d04-ad7b-01429e616d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:51.318155 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de5ba1-ffe6-4d04-ad7b-01429e616d66 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:51.318964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de5ba1-ffe6-4d04-ad7b-01429e616d66 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:51.320544 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de5ba1-ffe6-4d04-ad7b-01429e616d66 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-99de5ba1-ffe6-4d04-ad7b-01429e616d66 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:51.330016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:51.331000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:51.331886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:51.331886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:51.331886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:51.331886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:51.331886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:51.331886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:51.331886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:51.331886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:51.331886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:51.331886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:51.331886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:51.331886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:51.331886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 06:00:51.331886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:51.334683 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 469/930] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Aug 23 06:00:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:51.344049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a109e-74ea-4274-bed1-d52cc323bca1 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:51.374129 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de5ba1-ffe6-4d04-ad7b-01429e616d66 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:51.448691 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:51.453698 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a109e-74ea-4274-bed1-d52cc323bca1 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:51.484251 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de5ba1-ffe6-4d04-ad7b-01429e616d66 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:51.490815 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ed2cd-1c24-4aba-97ff-d41cb10f0db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:51.491989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ed2cd-1c24-4aba-97ff-d41cb10f0db2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:51.492443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ed2cd-1c24-4aba-97ff-d41cb10f0db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:51.492870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ed2cd-1c24-4aba-97ff-d41cb10f0db2 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:51.493943 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ed2cd-1c24-4aba-97ff-d41cb10f0db2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:51.494439 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ed2cd-1c24-4aba-97ff-d41cb10f0db2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:51.496781 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ed2cd-1c24-4aba-97ff-d41cb10f0db2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:51.508979 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 462/931] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 23 06:00:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:51.510542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ed2cd-1c24-4aba-97ff-d41cb10f0db2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:51.523016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:51.525697 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543151e7-1511-43bd-a790-c6b1ae6c6adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:51.526590 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543151e7-1511-43bd-a790-c6b1ae6c6adf None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:51.527056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543151e7-1511-43bd-a790-c6b1ae6c6adf None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:51.527574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543151e7-1511-43bd-a790-c6b1ae6c6adf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:51.542186 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543151e7-1511-43bd-a790-c6b1ae6c6adf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-543151e7-1511-43bd-a790-c6b1ae6c6adf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:51.549164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:51.550149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:51.551030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:51.551030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:51.551030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:51.551030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:51.551030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:51.551030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:51.551030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:51.551030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:51.551030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:51.551030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:51.551030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 06:00:51.551030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:51.551685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 470/932] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 23 06:00:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:51.558814 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543151e7-1511-43bd-a790-c6b1ae6c6adf None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:51.642983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ed2cd-1c24-4aba-97ff-d41cb10f0db2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:51.680089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543151e7-1511-43bd-a790-c6b1ae6c6adf None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:51.693743 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cddc2-0823-4458-8ec7-eb71ac557027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:51.694923 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cddc2-0823-4458-8ec7-eb71ac557027 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:51.695353 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cddc2-0823-4458-8ec7-eb71ac557027 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:51.695766 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cddc2-0823-4458-8ec7-eb71ac557027 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:51.696721 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cddc2-0823-4458-8ec7-eb71ac557027 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:51.697568 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cddc2-0823-4458-8ec7-eb71ac557027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:51.699966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cddc2-0823-4458-8ec7-eb71ac557027 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:51.707380 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 463/933] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 23 06:00:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:51.715563 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cddc2-0823-4458-8ec7-eb71ac557027 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:51.718828 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:51.727807 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904837d-2f1d-4186-8aa3-1521ae0aaea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:51.728558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904837d-2f1d-4186-8aa3-1521ae0aaea3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:51.729045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904837d-2f1d-4186-8aa3-1521ae0aaea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:51.729709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904837d-2f1d-4186-8aa3-1521ae0aaea3 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:51.731074 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904837d-2f1d-4186-8aa3-1521ae0aaea3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:51.731738 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904837d-2f1d-4186-8aa3-1521ae0aaea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:51.733414 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904837d-2f1d-4186-8aa3-1521ae0aaea3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:51.745218 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 471/934] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 23 06:00:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:51.751051 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904837d-2f1d-4186-8aa3-1521ae0aaea3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:51.795446 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:51.835789 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cddc2-0823-4458-8ec7-eb71ac557027 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:51.862232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904837d-2f1d-4186-8aa3-1521ae0aaea3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:51.877554 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe0384-8e34-4978-8a22-e7f6c6fec57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:51.878445 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe0384-8e34-4978-8a22-e7f6c6fec57b None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:51.878877 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe0384-8e34-4978-8a22-e7f6c6fec57b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:51.879712 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe0384-8e34-4978-8a22-e7f6c6fec57b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:51.880668 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe0384-8e34-4978-8a22-e7f6c6fec57b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:51.882921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe0384-8e34-4978-8a22-e7f6c6fec57b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:51.905455 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493ba7e-4aa0-49d4-8b2a-78cc8f405d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:51.906344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493ba7e-4aa0-49d4-8b2a-78cc8f405d51 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:51.906804 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493ba7e-4aa0-49d4-8b2a-78cc8f405d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:51.907293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493ba7e-4aa0-49d4-8b2a-78cc8f405d51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:51.914023 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 464/935] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 23 06:00:51 2022] POST /identity/v3/endpoints => generated 347 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 06:00:51.926774 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493ba7e-4aa0-49d4-8b2a-78cc8f405d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:51.931215 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493ba7e-4aa0-49d4-8b2a-78cc8f405d51 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-5493ba7e-4aa0-49d4-8b2a-78cc8f405d51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:51.949283 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:51.950365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:51.951267 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:51.951267 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:51.951267 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:51.951267 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 472/936] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:51.957684 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe0384-8e34-4978-8a22-e7f6c6fec57b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:52.092505 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe0384-8e34-4978-8a22-e7f6c6fec57b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:52.112422 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:52.133708 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c3dc3-2104-4d51-a0d2-af25992f458c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:52.134585 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c3dc3-2104-4d51-a0d2-af25992f458c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:52.135066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c3dc3-2104-4d51-a0d2-af25992f458c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:52.135487 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c3dc3-2104-4d51-a0d2-af25992f458c None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:52.136325 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c3dc3-2104-4d51-a0d2-af25992f458c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:52.136799 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c3dc3-2104-4d51-a0d2-af25992f458c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:52.138493 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c3dc3-2104-4d51-a0d2-af25992f458c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:52.153088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 465/937] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:52.155444 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ce4a4-c674-4cbe-9aab-156cff4a3ad2 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:52.155726 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ce4a4-c674-4cbe-9aab-156cff4a3ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:52.155995 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ce4a4-c674-4cbe-9aab-156cff4a3ad2 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:52.157648 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 473/938] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:52.158958 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c3dc3-2104-4d51-a0d2-af25992f458c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:52.162412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2e8f3-7aa0-45e2-8bfd-fd1b21afb636 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:52.162915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2e8f3-7aa0-45e2-8bfd-fd1b21afb636 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:52.163015 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2e8f3-7aa0-45e2-8bfd-fd1b21afb636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:52.183816 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-cbc2e8f3-7aa0-45e2-8bfd-fd1b21afb636 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:52.185004 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-cbc2e8f3-7aa0-45e2-8bfd-fd1b21afb636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:52.232838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:52.268668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 474/939] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:00:52.304133 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c3dc3-2104-4d51-a0d2-af25992f458c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:52.343294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456e841-0806-4577-a1a9-668da2784324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:52.344184 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456e841-0806-4577-a1a9-668da2784324 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:52.344621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456e841-0806-4577-a1a9-668da2784324 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:52.345354 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456e841-0806-4577-a1a9-668da2784324 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:52.353951 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b456e841-0806-4577-a1a9-668da2784324 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:52.356288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b456e841-0806-4577-a1a9-668da2784324 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-b456e841-0806-4577-a1a9-668da2784324 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:52.369243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:52.370426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:52.371922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:52.371922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:52.371922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:52.371922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:52.371922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:52.371922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:52.371922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:52.371922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:52.371922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 06:00:52.371922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:52.374139 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 466/940] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 23 06:00:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:52.379689 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2e8f3-7aa0-45e2-8bfd-fd1b21afb636 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:52.485769 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2e8f3-7aa0-45e2-8bfd-fd1b21afb636 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:52.522908 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f8b13-7b6b-4535-970a-844b6bdbf49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:52.523510 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f8b13-7b6b-4535-970a-844b6bdbf49d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:52.523783 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f8b13-7b6b-4535-970a-844b6bdbf49d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:52.524048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f8b13-7b6b-4535-970a-844b6bdbf49d None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:52.524771 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f8b13-7b6b-4535-970a-844b6bdbf49d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:52.525069 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f8b13-7b6b-4535-970a-844b6bdbf49d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:52.526549 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f8b13-7b6b-4535-970a-844b6bdbf49d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:52.542798 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456e841-0806-4577-a1a9-668da2784324 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:52.544795 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 475/941] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 23 06:00:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:52.550251 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f8b13-7b6b-4535-970a-844b6bdbf49d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:52.586835 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:52.598719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:52.665949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f8b13-7b6b-4535-970a-844b6bdbf49d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:52.670709 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456e841-0806-4577-a1a9-668da2784324 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:52.703721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad4f8c-246b-4a21-84e2-e459d1e4eb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:52.704360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad4f8c-246b-4a21-84e2-e459d1e4eb90 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:52.704639 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad4f8c-246b-4a21-84e2-e459d1e4eb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:52.704910 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad4f8c-246b-4a21-84e2-e459d1e4eb90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:52.711412 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1f75b-c5b3-4407-acfe-710de0b747d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:52.712379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1f75b-c5b3-4407-acfe-710de0b747d7 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:52.712861 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1f75b-c5b3-4407-acfe-710de0b747d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:52.713320 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1f75b-c5b3-4407-acfe-710de0b747d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:52.714211 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1f75b-c5b3-4407-acfe-710de0b747d7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:52.716438 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad4f8c-246b-4a21-84e2-e459d1e4eb90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:52.716721 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1f75b-c5b3-4407-acfe-710de0b747d7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:52.717880 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad4f8c-246b-4a21-84e2-e459d1e4eb90 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:52.720115 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: WARNING keystone.server.flask.common [None req-87b1f75b-c5b3-4407-acfe-710de0b747d7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-e0ad4f8c-246b-4a21-84e2-e459d1e4eb90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:52.734045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:52.735362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:52.736824 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:52.736824 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:52.736824 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:52.736824 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 476/942] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:52.736824 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e4f97-3844-4916-a601-4bcd05d57349 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:52.736824 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e4f97-3844-4916-a601-4bcd05d57349 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:52.737189 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e4f97-3844-4916-a601-4bcd05d57349 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:52.738544 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 477/943] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:52.740584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e4f97-3844-4916-a601-4bcd05d57349 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:52.743956 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.sql.core [None req-87b1f75b-c5b3-4407-acfe-710de0b747d7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 06:00:52.743956 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 23 06:00:52.743956 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [parameters: {'id': '66b19bc7ae3544269450e7bacacc9f8f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 23 06:00:52.743956 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81746) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 06:00:52.760929 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-87b1f75b-c5b3-4407-acfe-710de0b747d7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 66b19bc7ae3544269450e7bacacc9f8f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 66b19bc7ae3544269450e7bacacc9f8f. Aug 23 06:00:52.760929 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:52.760929 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 06:00:52.760929 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 06:00:52.760929 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 06:00:52.760929 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 06:00:52.760929 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 06:00:52.760929 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 06:00:52.760929 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 06:00:52.760929 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application conn.query(q) Aug 23 06:00:52.760929 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 06:00:52.760929 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 06:00:52.760929 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result.read() Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 23 06:00:52.762957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application next(self.gen) Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application yield resource Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application next(self.gen) Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise self.value Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application session.commit() Aug 23 06:00:52.764392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.session.flush() Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application compat.raise_( Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 06:00:52.765322 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 06:00:52.766248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 06:00:52.766248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 06:00:52.766248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 06:00:52.766248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 06:00:52.766248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 23 06:00:52.766248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 23 06:00:52.766248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 06:00:52.766248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 06:00:52.766248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 06:00:52.766248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 06:00:52.766248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 06:00:52.766248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 06:00:52.766248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 06:00:52.766248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 06:00:52.766248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application conn.query(q) Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result.read() Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 06:00:52.768871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 06:00:52.771634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 06:00:52.771634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 06:00:52.771634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 06:00:52.771634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 06:00:52.771634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 06:00:52.771634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 06:00:52.771634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 06:00:52.771634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 23 06:00:52.771634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [parameters: {'id': '66b19bc7ae3544269450e7bacacc9f8f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 23 06:00:52.771634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 06:00:52.771634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:52.771634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:52.771634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:52.771634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:52.771634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:52.773970 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:52.776595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:52.776595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 23 06:00:52.776595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 23 06:00:52.776595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:52.776595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:52.776595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 23 06:00:52.776595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 23 06:00:52.776595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 66b19bc7ae3544269450e7bacacc9f8f. Aug 23 06:00:52.776595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:52.776595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 467/944] 10.222.0.20 () {60 vars in 1140 bytes} [Tue Aug 23 06:00:52 2022] POST /identity/v3/projects => generated 262 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:52.776595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1136cb6-e8c9-45c0-8de8-1927fbb92816 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:52.776595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1136cb6-e8c9-45c0-8de8-1927fbb92816 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:52.779923 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1136cb6-e8c9-45c0-8de8-1927fbb92816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:52.790567 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e1136cb6-e8c9-45c0-8de8-1927fbb92816 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:52.792181 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e1136cb6-e8c9-45c0-8de8-1927fbb92816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:52.844528 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:52.863992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e4f97-3844-4916-a601-4bcd05d57349 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:52.873512 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 468/945] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:52.873512 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1136cb6-e8c9-45c0-8de8-1927fbb92816 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:52.900655 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b03af-c077-4859-9ff1-7f267624cc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:52.901273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b03af-c077-4859-9ff1-7f267624cc29 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:52.901553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b03af-c077-4859-9ff1-7f267624cc29 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:52.901823 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b03af-c077-4859-9ff1-7f267624cc29 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:52.903112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b03af-c077-4859-9ff1-7f267624cc29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:52.903674 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b03af-c077-4859-9ff1-7f267624cc29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:52.904800 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b03af-c077-4859-9ff1-7f267624cc29 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:52.906691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:52.929701 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 478/946] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:52.930655 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b03af-c077-4859-9ff1-7f267624cc29 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:53.024368 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1136cb6-e8c9-45c0-8de8-1927fbb92816 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:53.058032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b03af-c077-4859-9ff1-7f267624cc29 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:53.071830 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b8450-7b13-483c-ad71-e4eb5d50f865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:53.072709 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b8450-7b13-483c-ad71-e4eb5d50f865 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:53.073447 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b8450-7b13-483c-ad71-e4eb5d50f865 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:53.073731 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b8450-7b13-483c-ad71-e4eb5d50f865 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:53.083988 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b8450-7b13-483c-ad71-e4eb5d50f865 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:53.086173 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b8450-7b13-483c-ad71-e4eb5d50f865 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:53.094825 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab55fec-f580-445c-a8e6-2986f57421aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:53.095576 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab55fec-f580-445c-a8e6-2986f57421aa None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:53.095914 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab55fec-f580-445c-a8e6-2986f57421aa None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:53.096248 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab55fec-f580-445c-a8e6-2986f57421aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:53.100745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-dd8b8450-7b13-483c-ad71-e4eb5d50f865 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:53.100745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:53.100745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:53.100745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:53.100745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:53.100745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:53.100745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:53.100745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:53.100745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:53.100745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:53.100745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:53.100745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:53.100745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:53.100745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:53.100745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:53.101612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:53.102495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:53.102495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:53.102495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:53.102495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:53.102495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:53.102495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:53.102495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:53.102495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:53.103554 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 469/947] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:53.108444 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b8450-7b13-483c-ad71-e4eb5d50f865 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:53.109505 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab55fec-f580-445c-a8e6-2986f57421aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:53.110985 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab55fec-f580-445c-a8e6-2986f57421aa None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:53.123162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-1ab55fec-f580-445c-a8e6-2986f57421aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:53.123162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:53.123162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:53.123162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:53.123162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:53.123162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:53.123162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:53.123162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:53.123162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:53.123162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:53.123162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:53.123162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:53.123162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:53.123162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:53.123162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:53.124060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:53.124978 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:53.124978 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:53.124978 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:53.124978 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:53.124978 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:53.124978 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:53.124978 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:53.124978 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:53.128962 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 479/948] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:53.129984 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab55fec-f580-445c-a8e6-2986f57421aa None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:53.134612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:53.216257 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:53.235332 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b8450-7b13-483c-ad71-e4eb5d50f865 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:53.255923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab55fec-f580-445c-a8e6-2986f57421aa None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:53.275710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c14fb-03f9-4236-8d09-d30fc5c7e5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:53.276538 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c14fb-03f9-4236-8d09-d30fc5c7e5ec None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:53.277719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c14fb-03f9-4236-8d09-d30fc5c7e5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:53.278496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c14fb-03f9-4236-8d09-d30fc5c7e5ec None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:53.279196 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c14fb-03f9-4236-8d09-d30fc5c7e5ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:53.279511 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c14fb-03f9-4236-8d09-d30fc5c7e5ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:53.281202 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c14fb-03f9-4236-8d09-d30fc5c7e5ec None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:53.292130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4956f-0164-4469-b5b8-f35530b5cc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:53.293366 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4956f-0164-4469-b5b8-f35530b5cc47 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:53.293753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4956f-0164-4469-b5b8-f35530b5cc47 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:53.294239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4956f-0164-4469-b5b8-f35530b5cc47 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:53.295607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b4956f-0164-4469-b5b8-f35530b5cc47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:53.295918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b4956f-0164-4469-b5b8-f35530b5cc47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:53.297358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b4956f-0164-4469-b5b8-f35530b5cc47 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:53.314609 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 470/949] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 23 06:00:53 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:53.314609 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c14fb-03f9-4236-8d09-d30fc5c7e5ec None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:53.328569 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 480/950] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:53.329795 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4956f-0164-4469-b5b8-f35530b5cc47 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:53.411888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:53.460981 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c14fb-03f9-4236-8d09-d30fc5c7e5ec None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:53.461715 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4956f-0164-4469-b5b8-f35530b5cc47 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:53.508098 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffb9d5-3c04-4a51-b2a2-b912a029162d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:53.508737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffb9d5-3c04-4a51-b2a2-b912a029162d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:53.509013 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffb9d5-3c04-4a51-b2a2-b912a029162d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:53.509306 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffb9d5-3c04-4a51-b2a2-b912a029162d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:53.509856 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffb9d5-3c04-4a51-b2a2-b912a029162d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:53.511258 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffb9d5-3c04-4a51-b2a2-b912a029162d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:53.511368 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377203b4-3f9c-4582-8905-1870c3fa2db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:53.512572 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377203b4-3f9c-4582-8905-1870c3fa2db6 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:53.512858 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377203b4-3f9c-4582-8905-1870c3fa2db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:53.513103 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377203b4-3f9c-4582-8905-1870c3fa2db6 None admin] PATH_INFO: `/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/078a012fea464954b0632a6ce86d7cec/roles/aa12d72368054e13a59fa2a814d51afd` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:53.515650 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377203b4-3f9c-4582-8905-1870c3fa2db6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0b43dead1664475b9dd52113e08d7ed, user_id=078a012fea464954b0632a6ce86d7cec, role_id=aa12d72368054e13a59fa2a814d51afd)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:53.517060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377203b4-3f9c-4582-8905-1870c3fa2db6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-a9ffb9d5-3c04-4a51-b2a2-b912a029162d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:53.522190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:53.523542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:53.524423 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:53.524423 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:53.524423 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:53.524423 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:53.524423 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:53.524423 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:53.524423 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:53.524423 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:53.524423 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:53.524423 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:53.524423 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:53.524423 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:53.524423 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 06:00:53.524423 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:53.529474 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 471/951] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 23 06:00:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:53.531384 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffb9d5-3c04-4a51-b2a2-b912a029162d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:53.540783 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 481/952] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:00:53 2022] PUT /identity/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/078a012fea464954b0632a6ce86d7cec/roles/aa12d72368054e13a59fa2a814d51afd => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:00:53.546324 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:53.676893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffb9d5-3c04-4a51-b2a2-b912a029162d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:53.719326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009bc27-1eb2-4166-9c8b-415201130c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:53.719945 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009bc27-1eb2-4166-9c8b-415201130c41 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:53.720217 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009bc27-1eb2-4166-9c8b-415201130c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:53.720485 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009bc27-1eb2-4166-9c8b-415201130c41 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:53.721115 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009bc27-1eb2-4166-9c8b-415201130c41 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:53.721411 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009bc27-1eb2-4166-9c8b-415201130c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:53.722798 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009bc27-1eb2-4166-9c8b-415201130c41 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:53.728370 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 472/953] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 23 06:00:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:53.734925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377203b4-3f9c-4582-8905-1870c3fa2db6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:53.845191 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377203b4-3f9c-4582-8905-1870c3fa2db6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:53.871201 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:53.885808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d591f2-4f44-487a-a8f6-15886c73618d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:53.886424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d591f2-4f44-487a-a8f6-15886c73618d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:53.887267 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d591f2-4f44-487a-a8f6-15886c73618d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:53.887566 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d591f2-4f44-487a-a8f6-15886c73618d None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:53.888162 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d591f2-4f44-487a-a8f6-15886c73618d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:53.888515 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d591f2-4f44-487a-a8f6-15886c73618d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:53.890058 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d591f2-4f44-487a-a8f6-15886c73618d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:53.896042 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 482/954] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 23 06:00:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:53.901664 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009bc27-1eb2-4166-9c8b-415201130c41 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:53.982913 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:54.019375 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009bc27-1eb2-4166-9c8b-415201130c41 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:54.054648 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76051bf9-85d1-436b-b9d3-7ef68d910cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:54.055254 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76051bf9-85d1-436b-b9d3-7ef68d910cb9 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:54.055526 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76051bf9-85d1-436b-b9d3-7ef68d910cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:54.055792 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76051bf9-85d1-436b-b9d3-7ef68d910cb9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:54.056462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76051bf9-85d1-436b-b9d3-7ef68d910cb9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:54.057822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76051bf9-85d1-436b-b9d3-7ef68d910cb9 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:54.080152 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 473/955] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 23 06:00:53 2022] POST /identity/v3/endpoints => generated 349 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:00:54.111624 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b9a75-cca5-4e08-be5d-7dcd5d7ff689 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:54.112028 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b9a75-cca5-4e08-be5d-7dcd5d7ff689 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:54.112222 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b9a75-cca5-4e08-be5d-7dcd5d7ff689 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:54.113541 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 483/956] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:54.119858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016c05e-17f6-4ca4-a86b-41fecf1bee32 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:54.120224 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016c05e-17f6-4ca4-a86b-41fecf1bee32 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:54.120562 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016c05e-17f6-4ca4-a86b-41fecf1bee32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:54.142501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1016c05e-17f6-4ca4-a86b-41fecf1bee32 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:54.143895 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1016c05e-17f6-4ca4-a86b-41fecf1bee32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:54.235018 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 474/957] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:54.499049 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b9a75-cca5-4e08-be5d-7dcd5d7ff689 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:54.609898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b9a75-cca5-4e08-be5d-7dcd5d7ff689 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:54.648558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c421c-d69b-48eb-919f-311f0771b5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:54.649198 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c421c-d69b-48eb-919f-311f0771b5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:54.649465 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c421c-d69b-48eb-919f-311f0771b5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:54.651914 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c421c-d69b-48eb-919f-311f0771b5d8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:54.652571 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c421c-d69b-48eb-919f-311f0771b5d8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:54.654210 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c421c-d69b-48eb-919f-311f0771b5d8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-3e6c421c-d69b-48eb-919f-311f0771b5d8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:54.665923 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:54.668140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:54.669653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:54.669653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:54.669653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:54.669653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:54.669653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:54.669653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:54.669653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:54.669653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:54.669653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:54.669653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:54.669653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:54.669653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:54.669653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 06:00:54.669653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:54.669653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 484/958] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:54.675982 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016c05e-17f6-4ca4-a86b-41fecf1bee32 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:54.733946 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:54.785889 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016c05e-17f6-4ca4-a86b-41fecf1bee32 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:54.821858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e10987-5f2a-4ccf-9a6f-8c7547f84593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:54.822470 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e10987-5f2a-4ccf-9a6f-8c7547f84593 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:54.822774 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e10987-5f2a-4ccf-9a6f-8c7547f84593 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:54.823071 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e10987-5f2a-4ccf-9a6f-8c7547f84593 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:54.823696 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e10987-5f2a-4ccf-9a6f-8c7547f84593 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:54.824173 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e10987-5f2a-4ccf-9a6f-8c7547f84593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:54.825588 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e10987-5f2a-4ccf-9a6f-8c7547f84593 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:54.831047 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 475/959] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 23 06:00:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:54.836799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c421c-d69b-48eb-919f-311f0771b5d8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:54.895485 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:54.949871 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c421c-d69b-48eb-919f-311f0771b5d8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:54.986006 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297cc53-826d-45c6-96b4-639883eab223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:54.986655 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297cc53-826d-45c6-96b4-639883eab223 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:54.986972 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297cc53-826d-45c6-96b4-639883eab223 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:54.987247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297cc53-826d-45c6-96b4-639883eab223 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:54.987886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297cc53-826d-45c6-96b4-639883eab223 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:54.988280 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297cc53-826d-45c6-96b4-639883eab223 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:54.989740 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297cc53-826d-45c6-96b4-639883eab223 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:54.995520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 485/960] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 23 06:00:54 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:55.456584 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd4322-d975-4d7b-a572-836fd175978d None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:55.456972 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd4322-d975-4d7b-a572-836fd175978d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:55.457148 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd4322-d975-4d7b-a572-836fd175978d None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:55.458774 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 476/961] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:55.464076 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8cc19-2b49-408e-9fa9-ee7d0b57be66 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:55.464362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8cc19-2b49-408e-9fa9-ee7d0b57be66 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:55.464671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8cc19-2b49-408e-9fa9-ee7d0b57be66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:55.483592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-32a8cc19-2b49-408e-9fa9-ee7d0b57be66 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:55.484773 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-32a8cc19-2b49-408e-9fa9-ee7d0b57be66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:55.556110 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 486/962] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:55.663864 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb70173-290c-4648-aa2b-e55efadcc191 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:55.664330 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb70173-290c-4648-aa2b-e55efadcc191 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:55.664705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb70173-290c-4648-aa2b-e55efadcc191 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:55.666484 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 477/963] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:55.672023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb522f02-c5df-4d19-91c8-762a4a46f718 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:55.672415 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb522f02-c5df-4d19-91c8-762a4a46f718 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:55.672803 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb522f02-c5df-4d19-91c8-762a4a46f718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:55.692691 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-bb522f02-c5df-4d19-91c8-762a4a46f718 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:55.693868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-bb522f02-c5df-4d19-91c8-762a4a46f718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:55.738687 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:55.770651 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 487/964] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:55.832314 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb70173-290c-4648-aa2b-e55efadcc191 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:55.946211 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb70173-290c-4648-aa2b-e55efadcc191 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:55.965430 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:55.983826 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d8183-ccb6-4dba-aff7-15231310f56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:55.984427 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d8183-ccb6-4dba-aff7-15231310f56f None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:55.984756 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d8183-ccb6-4dba-aff7-15231310f56f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:55.985030 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d8183-ccb6-4dba-aff7-15231310f56f None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:55.985664 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d8183-ccb6-4dba-aff7-15231310f56f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:55.987131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d8183-ccb6-4dba-aff7-15231310f56f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:56.006563 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.sql.core [None req-307d8183-ccb6-4dba-aff7-15231310f56f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 06:00:56.006563 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 23 06:00:56.006563 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [parameters: {'user_id': '941a20340b4e450cb9b416f57166d964', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 23 06:00:56.006563 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81746) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 06:00:56.013260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-307d8183-ccb6-4dba-aff7-15231310f56f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 23 06:00:56.013260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:56.013260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 06:00:56.013260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 06:00:56.013260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 06:00:56.013260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 06:00:56.013260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 06:00:56.013260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 06:00:56.013260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 06:00:56.013260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application conn.query(q) Aug 23 06:00:56.013260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 06:00:56.013260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 06:00:56.013260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 06:00:56.013260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result.read() Aug 23 06:00:56.013260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 06:00:56.014315 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application next(self.gen) Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application yield resource Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application next(self.gen) Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise self.value Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application session.commit() Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 06:00:56.015513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.session.flush() Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application compat.raise_( Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 06:00:56.016435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 06:00:56.017310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application conn.query(q) Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application result.read() Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 06:00:56.018233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 06:00:56.019175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 06:00:56.019175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 06:00:56.019175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 06:00:56.019175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 06:00:56.019175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 06:00:56.019175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 23 06:00:56.019175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [parameters: {'user_id': '941a20340b4e450cb9b416f57166d964', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 23 06:00:56.019175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 06:00:56.019175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:56.019175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:56.019175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:56.019175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:56.019175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:56.019175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:56.019175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 23 06:00:56.020127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 23 06:00:56.021138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 23 06:00:56.021138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 23 06:00:56.021138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 06:00:56.021138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 06:00:56.021138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 23 06:00:56.021138 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:56.023184 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 478/965] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Aug 23 06:00:55 2022] POST /identity/v3/users => generated 160 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:56.031191 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb522f02-c5df-4d19-91c8-762a4a46f718 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:56.049571 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d8183-ccb6-4dba-aff7-15231310f56f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:56.164422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb522f02-c5df-4d19-91c8-762a4a46f718 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:56.165155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d8183-ccb6-4dba-aff7-15231310f56f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:56.204326 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f7f63-2302-4963-9d0c-399a227e1821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:56.204789 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a6a77-ae52-403f-8b85-40f64442134e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:56.205040 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f7f63-2302-4963-9d0c-399a227e1821 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:56.205224 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f7f63-2302-4963-9d0c-399a227e1821 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:56.205407 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a6a77-ae52-403f-8b85-40f64442134e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:56.205507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f7f63-2302-4963-9d0c-399a227e1821 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:56.205674 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a6a77-ae52-403f-8b85-40f64442134e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:56.205995 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a6a77-ae52-403f-8b85-40f64442134e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:56.212854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f7f63-2302-4963-9d0c-399a227e1821 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:56.214280 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f7f63-2302-4963-9d0c-399a227e1821 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:56.219076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a6a77-ae52-403f-8b85-40f64442134e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-e24a6a77-ae52-403f-8b85-40f64442134e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:00:56.224066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:56.225052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:56.225975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:56.225975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:56.225975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:56.225975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:56.225975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:00:56.225975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:00:56.225975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:00:56.225975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:00:56.225975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:00:56.225975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:00:56.225975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 06:00:56.225975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:56.225975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 479/966] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 23 06:00:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:56.227277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2f44e-e68d-4793-a763-07fc576fa770 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:56.227277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2f44e-e68d-4793-a763-07fc576fa770 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:56.227277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2f44e-e68d-4793-a763-07fc576fa770 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-996f7f63-2302-4963-9d0c-399a227e1821 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:56.231530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:56.232469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:56.240117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:56.240117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:56.240117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:56.240117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:56.240117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:56.240117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:56.240117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:56.240117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:56.240117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 06:00:56.240117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:56.241457 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 480/968] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:56.241457 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2f44e-e68d-4793-a763-07fc576fa770 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:56.241752 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 488/968] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:00:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:56.241752 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db83dc2-a32a-4653-8fb5-21a4348de7a4 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:56.241752 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db83dc2-a32a-4653-8fb5-21a4348de7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:56.241752 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db83dc2-a32a-4653-8fb5-21a4348de7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:56.258667 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-5db83dc2-a32a-4653-8fb5-21a4348de7a4 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:56.259767 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5db83dc2-a32a-4653-8fb5-21a4348de7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:56.287311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:56.291734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:56.332343 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 489/969] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:56.333344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db83dc2-a32a-4653-8fb5-21a4348de7a4 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:56.362247 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2f44e-e68d-4793-a763-07fc576fa770 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:56.400982 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67524431-8ceb-40e7-824f-f7c032209f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:56.402892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67524431-8ceb-40e7-824f-f7c032209f36 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:56.403391 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67524431-8ceb-40e7-824f-f7c032209f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:56.403720 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67524431-8ceb-40e7-824f-f7c032209f36 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:56.404403 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67524431-8ceb-40e7-824f-f7c032209f36 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:56.404707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67524431-8ceb-40e7-824f-f7c032209f36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:56.406211 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67524431-8ceb-40e7-824f-f7c032209f36 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:56.426686 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 481/970] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 23 06:00:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:56.433844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67524431-8ceb-40e7-824f-f7c032209f36 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:56.470414 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db83dc2-a32a-4653-8fb5-21a4348de7a4 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:56.510976 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209a64d-beda-4ad8-b709-126cd0c970c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:56.512542 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209a64d-beda-4ad8-b709-126cd0c970c3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:56.512830 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209a64d-beda-4ad8-b709-126cd0c970c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:56.513100 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209a64d-beda-4ad8-b709-126cd0c970c3 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:56.513807 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209a64d-beda-4ad8-b709-126cd0c970c3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:56.514165 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209a64d-beda-4ad8-b709-126cd0c970c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:56.516181 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209a64d-beda-4ad8-b709-126cd0c970c3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:56.564612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 490/971] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Aug 23 06:00:56 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:56.582093 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67524431-8ceb-40e7-824f-f7c032209f36 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:56.584364 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:56.621847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209a64d-beda-4ad8-b709-126cd0c970c3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:56.625258 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0c393-8193-490d-89b2-5c398f272181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:56.625913 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0c393-8193-490d-89b2-5c398f272181 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:56.626187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0c393-8193-490d-89b2-5c398f272181 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:56.626451 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0c393-8193-490d-89b2-5c398f272181 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:56.642331 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0c393-8193-490d-89b2-5c398f272181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:56.644403 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0c393-8193-490d-89b2-5c398f272181 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-4be0c393-8193-490d-89b2-5c398f272181 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:56.662892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:56.664032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:56.664935 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:56.664935 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:56.664935 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:56.664935 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 482/972] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:56.671029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:56.671907 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0c393-8193-490d-89b2-5c398f272181 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:56.788296 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209a64d-beda-4ad8-b709-126cd0c970c3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:56.821857 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0c393-8193-490d-89b2-5c398f272181 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:56.833558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d535a-d1cc-4ea9-b4fc-30dbbb466a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:56.833558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d535a-d1cc-4ea9-b4fc-30dbbb466a10 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:56.833558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d535a-d1cc-4ea9-b4fc-30dbbb466a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:56.833558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d535a-d1cc-4ea9-b4fc-30dbbb466a10 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:56.834248 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d535a-d1cc-4ea9-b4fc-30dbbb466a10 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:56.835931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d535a-d1cc-4ea9-b4fc-30dbbb466a10 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-235d535a-d1cc-4ea9-b4fc-30dbbb466a10 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:56.846607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:56.847757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:56.848612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:56.848612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:56.848612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:56.848612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:56.848612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:56.848612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:56.848612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:56.848612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:56.848612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:56.848612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:56.848612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:00:56.848612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:00:56.848612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 06:00:56.848612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:56.852396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 491/973] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 23 06:00:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:56.854040 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d535a-d1cc-4ea9-b4fc-30dbbb466a10 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:56.882784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08f6f7-cd0e-48e7-9460-07bcff219edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:56.883530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08f6f7-cd0e-48e7-9460-07bcff219edd None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:56.883873 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08f6f7-cd0e-48e7-9460-07bcff219edd None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:56.884208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08f6f7-cd0e-48e7-9460-07bcff219edd None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:56.884992 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08f6f7-cd0e-48e7-9460-07bcff219edd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:56.885363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08f6f7-cd0e-48e7-9460-07bcff219edd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:56.887229 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08f6f7-cd0e-48e7-9460-07bcff219edd None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:56.902563 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 483/974] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:56.908699 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08f6f7-cd0e-48e7-9460-07bcff219edd None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:56.925380 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:56.988047 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:57.003424 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d535a-d1cc-4ea9-b4fc-30dbbb466a10 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:57.028242 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08f6f7-cd0e-48e7-9460-07bcff219edd None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:57.047430 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aec195-b80e-4d9b-a182-12c2f1505cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:57.047958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aec195-b80e-4d9b-a182-12c2f1505cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:57.048230 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aec195-b80e-4d9b-a182-12c2f1505cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:57.048499 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aec195-b80e-4d9b-a182-12c2f1505cb6 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:57.049154 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aec195-b80e-4d9b-a182-12c2f1505cb6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:57.049454 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aec195-b80e-4d9b-a182-12c2f1505cb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:57.051903 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aec195-b80e-4d9b-a182-12c2f1505cb6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:57.056626 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 492/975] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 23 06:00:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:57.057933 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bf720-8f41-4415-a4fd-9cc1bee61c0f None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:57.058902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bf720-8f41-4415-a4fd-9cc1bee61c0f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:57.059431 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bf720-8f41-4415-a4fd-9cc1bee61c0f None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:57.062548 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 493/976] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:57.063315 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bf720-8f41-4415-a4fd-9cc1bee61c0f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:57.070990 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fe327-09ec-468a-b57c-35a326255ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:57.071623 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fe327-09ec-468a-b57c-35a326255ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:57.072028 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fe327-09ec-468a-b57c-35a326255ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:57.072161 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fe327-09ec-468a-b57c-35a326255ab5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:57.080819 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fe327-09ec-468a-b57c-35a326255ab5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:57.082279 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fe327-09ec-468a-b57c-35a326255ab5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-f37fe327-09ec-468a-b57c-35a326255ab5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:00:57.094710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:57.095887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:57.097033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:57.097033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:57.097033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:00:57.097033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:00:57.097033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:00:57.097033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:00:57.097033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:00:57.097033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:00:57.097033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 06:00:57.097033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:00:57.099136 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 484/977] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 23 06:00:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:57.100917 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef5e00-0160-4f03-a115-c8b1ecdea6f0 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:57.101191 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef5e00-0160-4f03-a115-c8b1ecdea6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:57.101456 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef5e00-0160-4f03-a115-c8b1ecdea6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:57.121409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ecef5e00-0160-4f03-a115-c8b1ecdea6f0 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:57.123550 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ecef5e00-0160-4f03-a115-c8b1ecdea6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:57.184586 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bf720-8f41-4415-a4fd-9cc1bee61c0f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:57.203981 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 485/978] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:57.204858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef5e00-0160-4f03-a115-c8b1ecdea6f0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:57.212260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:57.228838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d1f77-b8ab-4716-9c94-8c86e921b378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:57.229456 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d1f77-b8ab-4716-9c94-8c86e921b378 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:57.229733 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d1f77-b8ab-4716-9c94-8c86e921b378 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:57.230002 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d1f77-b8ab-4716-9c94-8c86e921b378 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:57.231117 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d1f77-b8ab-4716-9c94-8c86e921b378 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:57.231334 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d1f77-b8ab-4716-9c94-8c86e921b378 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:57.232742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d1f77-b8ab-4716-9c94-8c86e921b378 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:57.245147 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 494/979] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 23 06:00:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:57.248287 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d1f77-b8ab-4716-9c94-8c86e921b378 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:57.271103 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:57.336736 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef5e00-0160-4f03-a115-c8b1ecdea6f0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:57.371571 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d1f77-b8ab-4716-9c94-8c86e921b378 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:57.384426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ed8ef-d8bb-46fa-b603-5f28698cb292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:57.384990 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ed8ef-d8bb-46fa-b603-5f28698cb292 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:57.385261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ed8ef-d8bb-46fa-b603-5f28698cb292 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:57.385524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ed8ef-d8bb-46fa-b603-5f28698cb292 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:57.386259 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ed8ef-d8bb-46fa-b603-5f28698cb292 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:57.387303 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ed8ef-d8bb-46fa-b603-5f28698cb292 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:57.388913 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ed8ef-d8bb-46fa-b603-5f28698cb292 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:57.415270 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 486/980] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Aug 23 06:00:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:57.422471 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ed8ef-d8bb-46fa-b603-5f28698cb292 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:57.427913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfbc20-616a-4ede-a89a-1b173c5d70ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:57.428552 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfbc20-616a-4ede-a89a-1b173c5d70ab None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:57.428871 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfbc20-616a-4ede-a89a-1b173c5d70ab None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:57.429143 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfbc20-616a-4ede-a89a-1b173c5d70ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:57.429928 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfbc20-616a-4ede-a89a-1b173c5d70ab None admin] RBAC: Adding query filter params (interface=admin, service_id=d6dac826bf7b4966a3d7f9cd97fd8598, region_id=RegionOne) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:57.430396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfbc20-616a-4ede-a89a-1b173c5d70ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:57.432035 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfbc20-616a-4ede-a89a-1b173c5d70ab None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:57.438998 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 495/981] 10.222.0.20 () {56 vars in 1249 bytes} [Tue Aug 23 06:00:57 2022] GET /identity/v3/endpoints?service_id=d6dac826bf7b4966a3d7f9cd97fd8598&interface=admin®ion_id=RegionOne => generated 193 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:57.445672 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfbc20-616a-4ede-a89a-1b173c5d70ab None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:57.503611 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:57.543770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:57.552222 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ed8ef-d8bb-46fa-b603-5f28698cb292 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:57.565757 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfbc20-616a-4ede-a89a-1b173c5d70ab None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:57.604649 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7a92c-b451-4547-9abf-c744ace7cd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:57.605783 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7a92c-b451-4547-9abf-c744ace7cd4b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:57.606121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7a92c-b451-4547-9abf-c744ace7cd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:57.606383 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7a92c-b451-4547-9abf-c744ace7cd4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:57.610695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0901ab-7d46-417a-b5da-8078f35f3b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:57.611323 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0901ab-7d46-417a-b5da-8078f35f3b32 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:57.611595 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0901ab-7d46-417a-b5da-8078f35f3b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:57.611866 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0901ab-7d46-417a-b5da-8078f35f3b32 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:57.612488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0901ab-7d46-417a-b5da-8078f35f3b32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:57.613897 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0901ab-7d46-417a-b5da-8078f35f3b32 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:57.623740 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 496/982] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 23 06:00:57 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 06:00:57.624866 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0901ab-7d46-417a-b5da-8078f35f3b32 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:57.707497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7a92c-b451-4547-9abf-c744ace7cd4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:57.709254 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7a92c-b451-4547-9abf-c744ace7cd4b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:57.794734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0901ab-7d46-417a-b5da-8078f35f3b32 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:57.805613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:57.844015 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d0c29-bce9-4fed-a9ec-344966589bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:57.844645 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d0c29-bce9-4fed-a9ec-344966589bb7 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:57.844920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d0c29-bce9-4fed-a9ec-344966589bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:57.845193 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d0c29-bce9-4fed-a9ec-344966589bb7 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:57.845850 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d0c29-bce9-4fed-a9ec-344966589bb7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:57.847362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d0c29-bce9-4fed-a9ec-344966589bb7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:57.865151 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 497/983] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 23 06:00:57 2022] POST /identity/v3/services => generated 242 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:00:57.876901 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 487/984] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:00:57 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 06:00:57.885722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d0c29-bce9-4fed-a9ec-344966589bb7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:57.899866 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:58.073857 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d0c29-bce9-4fed-a9ec-344966589bb7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:58.114200 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0e5f2-d18d-44c1-810f-b39c50736b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:58.114906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0e5f2-d18d-44c1-810f-b39c50736b28 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:58.115231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0e5f2-d18d-44c1-810f-b39c50736b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:58.115472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0e5f2-d18d-44c1-810f-b39c50736b28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:58.127296 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0e5f2-d18d-44c1-810f-b39c50736b28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:58.128738 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0e5f2-d18d-44c1-810f-b39c50736b28 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-dda0e5f2-d18d-44c1-810f-b39c50736b28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:00:58.144119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:58.145295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:00:58.146195 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:00:58.146195 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:00:58.146195 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:58.146195 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 498/985] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:00:58.151151 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7a92c-b451-4547-9abf-c744ace7cd4b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:58.262836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7a92c-b451-4547-9abf-c744ace7cd4b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:58.300898 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87564418-5e41-4b51-ae76-7f1af0dac74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:58.301509 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87564418-5e41-4b51-ae76-7f1af0dac74b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:58.301777 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87564418-5e41-4b51-ae76-7f1af0dac74b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:58.302037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87564418-5e41-4b51-ae76-7f1af0dac74b None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:58.302703 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87564418-5e41-4b51-ae76-7f1af0dac74b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:58.302994 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87564418-5e41-4b51-ae76-7f1af0dac74b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:58.304363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87564418-5e41-4b51-ae76-7f1af0dac74b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:58.321680 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 488/986] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:58.323221 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0e5f2-d18d-44c1-810f-b39c50736b28 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:58.385630 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:58.438939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0e5f2-d18d-44c1-810f-b39c50736b28 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:58.472788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe81ac-bc90-4954-a56d-d18dfd984bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:58.473415 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe81ac-bc90-4954-a56d-d18dfd984bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:58.473686 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe81ac-bc90-4954-a56d-d18dfd984bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:58.473948 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe81ac-bc90-4954-a56d-d18dfd984bc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:58.481425 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe81ac-bc90-4954-a56d-d18dfd984bc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:58.482871 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe81ac-bc90-4954-a56d-d18dfd984bc9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:58.494168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-b9fe81ac-bc90-4954-a56d-d18dfd984bc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:58.494168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:00:58.494168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:00:58.494168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:00:58.494168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:00:58.494168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:00:58.494168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:00:58.494168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:00:58.494168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:00:58.494168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:00:58.494168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:00:58.494168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:00:58.494168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:00:58.494168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:00:58.494168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:00:58.495185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:00:58.496888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:00:58.496888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:00:58.496888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:00:58.496888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:00:58.496888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:00:58.496888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:00:58.496888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:00:58.496888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:00:58.496888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 499/987] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:00:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:00:58.503802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87564418-5e41-4b51-ae76-7f1af0dac74b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:58.530154 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:58.614531 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87564418-5e41-4b51-ae76-7f1af0dac74b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:58.650333 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37bfed4-d16a-4331-b654-754c13b1aa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:58.650876 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37bfed4-d16a-4331-b654-754c13b1aa50 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:58.651149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37bfed4-d16a-4331-b654-754c13b1aa50 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:58.651416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37bfed4-d16a-4331-b654-754c13b1aa50 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:58.652621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37bfed4-d16a-4331-b654-754c13b1aa50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:58.652920 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37bfed4-d16a-4331-b654-754c13b1aa50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:58.654341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37bfed4-d16a-4331-b654-754c13b1aa50 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:58.670024 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 489/988] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:00:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:00:58.676163 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe81ac-bc90-4954-a56d-d18dfd984bc9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:00:58.783485 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe81ac-bc90-4954-a56d-d18dfd984bc9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:00:58.799442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:00:58.833768 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cfa11-be2c-4c31-9e0a-4a8973b37b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:00:58.834400 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cfa11-be2c-4c31-9e0a-4a8973b37b3a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:58.834722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cfa11-be2c-4c31-9e0a-4a8973b37b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:58.834992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cfa11-be2c-4c31-9e0a-4a8973b37b3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:58.835743 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cfa11-be2c-4c31-9e0a-4a8973b37b3a None admin] RBAC: Adding query filter params (role.id=aa12d72368054e13a59fa2a814d51afd, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=078a012fea464954b0632a6ce86d7cec) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:00:58.836047 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cfa11-be2c-4c31-9e0a-4a8973b37b3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:00:58.837507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cfa11-be2c-4c31-9e0a-4a8973b37b3a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:00:58.854664 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 500/989] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:00:58 2022] GET /identity/v3/role_assignments?user.id=078a012fea464954b0632a6ce86d7cec&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=aa12d72368054e13a59fa2a814d51afd => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:00:59.623869 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db110f16-0381-4ba3-9cf2-3238269155ae None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:59.624426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db110f16-0381-4ba3-9cf2-3238269155ae None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:59.624426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db110f16-0381-4ba3-9cf2-3238269155ae None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:59.625902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 490/990] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:59.630984 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f535f8-0482-4bf8-9024-49aebcfe4a5d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:59.631271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f535f8-0482-4bf8-9024-49aebcfe4a5d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:59.631548 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f535f8-0482-4bf8-9024-49aebcfe4a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:59.649847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-91f535f8-0482-4bf8-9024-49aebcfe4a5d None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:59.651074 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-91f535f8-0482-4bf8-9024-49aebcfe4a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:59.720008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 501/991] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:00:59.901590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa1c3b-4a38-4211-b662-8a22aae649a1 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:59.901840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa1c3b-4a38-4211-b662-8a22aae649a1 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:59.902390 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa1c3b-4a38-4211-b662-8a22aae649a1 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:59.904502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 491/992] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:00:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:00:59.909961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c01d93-5d68-4fed-83e1-4b6fb656924b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:00:59.910251 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c01d93-5d68-4fed-83e1-4b6fb656924b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:00:59.910540 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c01d93-5d68-4fed-83e1-4b6fb656924b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:00:59.930216 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-76c01d93-5d68-4fed-83e1-4b6fb656924b None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:00:59.931466 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-76c01d93-5d68-4fed-83e1-4b6fb656924b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:00:59.997044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa1c3b-4a38-4211-b662-8a22aae649a1 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:00.006864 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 502/993] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:00:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:00.091619 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:00.111611 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa1c3b-4a38-4211-b662-8a22aae649a1 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:00.147944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f0956-4679-48d6-85a2-16a47991c263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:00.148563 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f0956-4679-48d6-85a2-16a47991c263 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:00.148840 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f0956-4679-48d6-85a2-16a47991c263 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:00.149110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f0956-4679-48d6-85a2-16a47991c263 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:00.149684 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f0956-4679-48d6-85a2-16a47991c263 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:00.151079 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f0956-4679-48d6-85a2-16a47991c263 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-b04f0956-4679-48d6-85a2-16a47991c263 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:00.160637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:00.161681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:00.163116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:00.163116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:00.163116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:00.163116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:00.163116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:00.163116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:00.163116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:00.163116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:00.163116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:00.163116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:00.163116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:01:00.163116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:01:00.163116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 06:01:00.163116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:00.163116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 492/994] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Aug 23 06:00:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:00.167654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c01d93-5d68-4fed-83e1-4b6fb656924b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:00.203082 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:00.270118 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c01d93-5d68-4fed-83e1-4b6fb656924b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:00.275253 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f0956-4679-48d6-85a2-16a47991c263 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:00.305504 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658586c9-c046-4aae-beeb-7ed2b695ab52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:00.306060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658586c9-c046-4aae-beeb-7ed2b695ab52 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:00.306335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658586c9-c046-4aae-beeb-7ed2b695ab52 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:00.306643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658586c9-c046-4aae-beeb-7ed2b695ab52 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:00.307264 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658586c9-c046-4aae-beeb-7ed2b695ab52 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:00.307566 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658586c9-c046-4aae-beeb-7ed2b695ab52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:00.309500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658586c9-c046-4aae-beeb-7ed2b695ab52 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:00.314475 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 503/995] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 23 06:01:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:00.319968 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658586c9-c046-4aae-beeb-7ed2b695ab52 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:00.379879 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f0956-4679-48d6-85a2-16a47991c263 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:00.414120 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7669d77-a1ac-475d-80e4-42957c6bd350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:00.414686 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7669d77-a1ac-475d-80e4-42957c6bd350 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:00.414986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7669d77-a1ac-475d-80e4-42957c6bd350 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:00.415252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7669d77-a1ac-475d-80e4-42957c6bd350 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:00.415804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7669d77-a1ac-475d-80e4-42957c6bd350 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:00.417158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7669d77-a1ac-475d-80e4-42957c6bd350 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-a7669d77-a1ac-475d-80e4-42957c6bd350 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:00.426839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:00.433155 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:00.434212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:00.434212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:00.434212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:00.434212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:00.434212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:00.434212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:00.434212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:00.434212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:00.434212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:00.434212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:00.434212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:01:00.434212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:01:00.434212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 06:01:00.434212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:00.434212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 493/996] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:01:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:01:00.435074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7669d77-a1ac-475d-80e4-42957c6bd350 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:00.437656 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:00.442090 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658586c9-c046-4aae-beeb-7ed2b695ab52 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:00.447853 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:00.482412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcbfd8-5e09-4f53-b11b-980f33c8ff36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:00.483133 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcbfd8-5e09-4f53-b11b-980f33c8ff36 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:00.483406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcbfd8-5e09-4f53-b11b-980f33c8ff36 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:00.483673 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcbfd8-5e09-4f53-b11b-980f33c8ff36 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:00.484299 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfcbfd8-5e09-4f53-b11b-980f33c8ff36 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:00.484592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfcbfd8-5e09-4f53-b11b-980f33c8ff36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:00.485958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfcbfd8-5e09-4f53-b11b-980f33c8ff36 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:00.491292 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 504/997] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Aug 23 06:01:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:00.496951 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcbfd8-5e09-4f53-b11b-980f33c8ff36 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:00.549595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7669d77-a1ac-475d-80e4-42957c6bd350 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:00.587365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26ec01-09a3-428e-89ce-dfaa29595e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:00.587967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26ec01-09a3-428e-89ce-dfaa29595e1b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:00.588237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26ec01-09a3-428e-89ce-dfaa29595e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:00.588499 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26ec01-09a3-428e-89ce-dfaa29595e1b None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:00.589123 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26ec01-09a3-428e-89ce-dfaa29595e1b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:00.589745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26ec01-09a3-428e-89ce-dfaa29595e1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:00.590984 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26ec01-09a3-428e-89ce-dfaa29595e1b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:00.596242 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 494/998] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 23 06:01:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:00.601594 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26ec01-09a3-428e-89ce-dfaa29595e1b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:00.602884 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcbfd8-5e09-4f53-b11b-980f33c8ff36 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:00.640600 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b00b65-99f0-4923-95a6-ce640f497035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:00.641217 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b00b65-99f0-4923-95a6-ce640f497035 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:00.641494 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b00b65-99f0-4923-95a6-ce640f497035 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:00.641763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b00b65-99f0-4923-95a6-ce640f497035 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:00.642418 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b00b65-99f0-4923-95a6-ce640f497035 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:00.643822 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b00b65-99f0-4923-95a6-ce640f497035 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:00.662456 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 505/999] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 23 06:01:00 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 06:01:00.691036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:00.723520 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26ec01-09a3-428e-89ce-dfaa29595e1b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:00.771297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f5a2d-f7df-4178-bda1-840f8f1d7696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:00.771981 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f5a2d-f7df-4178-bda1-840f8f1d7696 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:00.772263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f5a2d-f7df-4178-bda1-840f8f1d7696 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:00.772530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f5a2d-f7df-4178-bda1-840f8f1d7696 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:00.773166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f5a2d-f7df-4178-bda1-840f8f1d7696 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:00.773463 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f5a2d-f7df-4178-bda1-840f8f1d7696 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:00.774891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f5a2d-f7df-4178-bda1-840f8f1d7696 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:00.786980 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 495/1000] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 23 06:01:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:00.788097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b00b65-99f0-4923-95a6-ce640f497035 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:00.818129 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:00.859285 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf4c5f-a703-4684-9e75-73ff11364fed None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:00.859567 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf4c5f-a703-4684-9e75-73ff11364fed None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:00.859836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf4c5f-a703-4684-9e75-73ff11364fed None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:00.861172 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 496/1001] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:01:00.866159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe68304-bf36-445a-b96a-1ca6fde14fd0 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:00.866432 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe68304-bf36-445a-b96a-1ca6fde14fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:00.866732 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe68304-bf36-445a-b96a-1ca6fde14fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:00.887766 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-bfe68304-bf36-445a-b96a-1ca6fde14fd0 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:00.888999 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bfe68304-bf36-445a-b96a-1ca6fde14fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:00.903528 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b00b65-99f0-4923-95a6-ce640f497035 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:00.944755 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831baa6b-b659-4818-b63d-f775efb053eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:00.945377 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831baa6b-b659-4818-b63d-f775efb053eb None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:00.945657 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831baa6b-b659-4818-b63d-f775efb053eb None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:00.945928 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831baa6b-b659-4818-b63d-f775efb053eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:00.946672 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831baa6b-b659-4818-b63d-f775efb053eb None admin] RBAC: Adding query filter params (interface=public, service_id=b32825b60ab0432eada1e75712837f90, region_id=RegionOne) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:00.946981 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831baa6b-b659-4818-b63d-f775efb053eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:00.951773 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831baa6b-b659-4818-b63d-f775efb053eb None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:00.960608 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 506/1002] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 23 06:01:00 2022] GET /identity/v3/endpoints?service_id=b32825b60ab0432eada1e75712837f90&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:00.961208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831baa6b-b659-4818-b63d-f775efb053eb None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:00.982776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 497/1003] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:00 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:01.057552 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:01.072341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831baa6b-b659-4818-b63d-f775efb053eb None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:01.131566 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e5c27-e82f-4305-8044-8d58e2a6f0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:01.132705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e5c27-e82f-4305-8044-8d58e2a6f0de None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:01.133221 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e5c27-e82f-4305-8044-8d58e2a6f0de None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:01.133723 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e5c27-e82f-4305-8044-8d58e2a6f0de None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:01.134880 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e5c27-e82f-4305-8044-8d58e2a6f0de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:01.137548 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e5c27-e82f-4305-8044-8d58e2a6f0de None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:01.152901 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 507/1004] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 23 06:01:00 2022] GET /identity/v3/services => generated 1727 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 06:01:01.249630 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe68304-bf36-445a-b96a-1ca6fde14fd0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:01.323824 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:01.369782 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe68304-bf36-445a-b96a-1ca6fde14fd0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:01.408957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84841f1d-15bc-4706-8ca4-5a5c7328735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:01.409579 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84841f1d-15bc-4706-8ca4-5a5c7328735f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:01.409854 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84841f1d-15bc-4706-8ca4-5a5c7328735f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:01.410126 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84841f1d-15bc-4706-8ca4-5a5c7328735f None admin] PATH_INFO: `/v3/services/image` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:01.410719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84841f1d-15bc-4706-8ca4-5a5c7328735f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:01.412086 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84841f1d-15bc-4706-8ca4-5a5c7328735f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-84841f1d-15bc-4706-8ca4-5a5c7328735f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:01.421284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:01.422294 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:01.423149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:01.423149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:01.423149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:01.423149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:01.423149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:01.423149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:01.423149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:01.423149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:01.423149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:01.423149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:01.423149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:01:01.423149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:01:01.423149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 06:01:01.423149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:01.423149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 498/1005] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Aug 23 06:01:01 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:01.429810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e5c27-e82f-4305-8044-8d58e2a6f0de None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:01.532487 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e5c27-e82f-4305-8044-8d58e2a6f0de None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:01.565548 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af033cd0-f882-41fc-8fb6-d16a2dd671e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:01.566151 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af033cd0-f882-41fc-8fb6-d16a2dd671e8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:01.566477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af033cd0-f882-41fc-8fb6-d16a2dd671e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:01.566771 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af033cd0-f882-41fc-8fb6-d16a2dd671e8 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:01.567386 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af033cd0-f882-41fc-8fb6-d16a2dd671e8 None admin] RBAC: Adding query filter params (name=image) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:01.567682 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af033cd0-f882-41fc-8fb6-d16a2dd671e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:01.569034 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af033cd0-f882-41fc-8fb6-d16a2dd671e8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:01.574103 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 508/1006] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:01.579485 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84841f1d-15bc-4706-8ca4-5a5c7328735f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:01.684211 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84841f1d-15bc-4706-8ca4-5a5c7328735f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:01.711024 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:01.720637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fed33-6fa9-47e1-8a45-81ac2a3fb161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:01.721238 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fed33-6fa9-47e1-8a45-81ac2a3fb161 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:01.721513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fed33-6fa9-47e1-8a45-81ac2a3fb161 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:01.721784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fed33-6fa9-47e1-8a45-81ac2a3fb161 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:01.722403 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9fed33-6fa9-47e1-8a45-81ac2a3fb161 None admin] RBAC: Adding query filter params (type=image) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:01.722728 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9fed33-6fa9-47e1-8a45-81ac2a3fb161 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:01.724088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9fed33-6fa9-47e1-8a45-81ac2a3fb161 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:01.734834 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 499/1007] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:02.621090 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cec88-25f7-4894-95a5-d6317ccd32b8 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:02.621613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cec88-25f7-4894-95a5-d6317ccd32b8 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:02.621613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cec88-25f7-4894-95a5-d6317ccd32b8 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:02.623237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 509/1008] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:02.628488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba7cb7-66e5-4041-8055-5c0667779370 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:02.628764 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba7cb7-66e5-4041-8055-5c0667779370 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:02.629027 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba7cb7-66e5-4041-8055-5c0667779370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:02.647238 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-80ba7cb7-66e5-4041-8055-5c0667779370 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:02.648457 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-80ba7cb7-66e5-4041-8055-5c0667779370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:02.714571 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 500/1009] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:02 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:02.983224 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cec88-25f7-4894-95a5-d6317ccd32b8 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:03.002143 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:03.088032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cec88-25f7-4894-95a5-d6317ccd32b8 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:03.124008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad8f99-aa87-43ea-afd4-73fe7f22230a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:03.125485 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad8f99-aa87-43ea-afd4-73fe7f22230a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:03.125629 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad8f99-aa87-43ea-afd4-73fe7f22230a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:03.126031 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad8f99-aa87-43ea-afd4-73fe7f22230a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:03.127404 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad8f99-aa87-43ea-afd4-73fe7f22230a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:03.129032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad8f99-aa87-43ea-afd4-73fe7f22230a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:03.130824 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 510/1010] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:03.136880 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba7cb7-66e5-4041-8055-5c0667779370 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:03.140976 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9da56e-176f-43bb-94f5-4570f4007163 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:03.141257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9da56e-176f-43bb-94f5-4570f4007163 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:03.141531 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9da56e-176f-43bb-94f5-4570f4007163 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:03.148833 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 511/1011] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:01:03.148833 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c98f9b-2e92-4f3f-b4d5-72924ec19fed None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:03.148833 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c98f9b-2e92-4f3f-b4d5-72924ec19fed None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:03.149096 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c98f9b-2e92-4f3f-b4d5-72924ec19fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:03.168371 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-a0c98f9b-2e92-4f3f-b4d5-72924ec19fed None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:03.169796 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a0c98f9b-2e92-4f3f-b4d5-72924ec19fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:03.244205 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 512/1012] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:03.252667 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba7cb7-66e5-4041-8055-5c0667779370 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:03.278254 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:03.292298 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625cbe4d-b597-4b9c-83f5-36be9ff4f035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:03.293542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cbe4d-b597-4b9c-83f5-36be9ff4f035 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:03.293850 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cbe4d-b597-4b9c-83f5-36be9ff4f035 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:03.294125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625cbe4d-b597-4b9c-83f5-36be9ff4f035 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:03.295008 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625cbe4d-b597-4b9c-83f5-36be9ff4f035 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:03.296628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625cbe4d-b597-4b9c-83f5-36be9ff4f035 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:03.330546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 501/1013] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 23 06:01:03 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:01:03.510214 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c98f9b-2e92-4f3f-b4d5-72924ec19fed None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:03.526571 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:03.618421 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c98f9b-2e92-4f3f-b4d5-72924ec19fed None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:03.654677 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a733c4-831d-4437-9805-d96738abc53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:03.655337 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a733c4-831d-4437-9805-d96738abc53c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:03.655573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a733c4-831d-4437-9805-d96738abc53c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:03.655840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a733c4-831d-4437-9805-d96738abc53c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:03.656420 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a733c4-831d-4437-9805-d96738abc53c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:03.657804 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a733c4-831d-4437-9805-d96738abc53c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-11a733c4-831d-4437-9805-d96738abc53c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:03.666770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:03.668389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:03.669398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:03.669398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:03.669398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:03.669398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:03.669398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:03.669398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:03.669398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:03.669398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:03.669398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:03.669398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:03.669398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:01:03.669398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:01:03.669398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 06:01:03.669398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:03.669398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 513/1014] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:01:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:03.674599 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625cbe4d-b597-4b9c-83f5-36be9ff4f035 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:03.754809 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0d37b-cfe7-44ff-b9dd-55530d77048d None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:03.755121 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0d37b-cfe7-44ff-b9dd-55530d77048d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:03.755361 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0d37b-cfe7-44ff-b9dd-55530d77048d None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:03.756982 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 514/1015] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:01:03.762259 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c7d68-72de-47fc-b03f-977047756726 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:03.762490 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c7d68-72de-47fc-b03f-977047756726 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:03.762769 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c7d68-72de-47fc-b03f-977047756726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:03.781026 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-626c7d68-72de-47fc-b03f-977047756726 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:03.782213 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-626c7d68-72de-47fc-b03f-977047756726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:03.782333 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625cbe4d-b597-4b9c-83f5-36be9ff4f035 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:03.825585 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac687f61-a5af-4c8c-8a77-326851141cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:03.826193 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac687f61-a5af-4c8c-8a77-326851141cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:03.826471 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac687f61-a5af-4c8c-8a77-326851141cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:03.826776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac687f61-a5af-4c8c-8a77-326851141cd5 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:03.827525 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac687f61-a5af-4c8c-8a77-326851141cd5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:03.827826 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac687f61-a5af-4c8c-8a77-326851141cd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:03.829323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac687f61-a5af-4c8c-8a77-326851141cd5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:03.834935 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 502/1016] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 23 06:01:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:03.840955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac687f61-a5af-4c8c-8a77-326851141cd5 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:03.861775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 515/1017] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:01:03.885964 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:03.959911 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac687f61-a5af-4c8c-8a77-326851141cd5 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:03.997136 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef32b25-472c-4111-aea8-22afc5aa21d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:03.997735 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef32b25-472c-4111-aea8-22afc5aa21d5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:03.998013 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef32b25-472c-4111-aea8-22afc5aa21d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:03.998283 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef32b25-472c-4111-aea8-22afc5aa21d5 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:03.998945 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef32b25-472c-4111-aea8-22afc5aa21d5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:03.999246 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef32b25-472c-4111-aea8-22afc5aa21d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:04.000628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef32b25-472c-4111-aea8-22afc5aa21d5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:04.006950 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 503/1018] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Aug 23 06:01:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:04.013219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c7d68-72de-47fc-b03f-977047756726 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:04.024153 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:04.119815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c7d68-72de-47fc-b03f-977047756726 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:04.129407 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef32b25-472c-4111-aea8-22afc5aa21d5 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:04.162554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d72f3-2616-4c74-9c08-6e5c6c0642a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:04.163170 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d72f3-2616-4c74-9c08-6e5c6c0642a8 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:04.163462 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d72f3-2616-4c74-9c08-6e5c6c0642a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:04.163730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d72f3-2616-4c74-9c08-6e5c6c0642a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:04.164383 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d72f3-2616-4c74-9c08-6e5c6c0642a8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:04.165743 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d72f3-2616-4c74-9c08-6e5c6c0642a8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:04.187605 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 516/1019] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 23 06:01:04 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 06:01:04.251363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef32b25-472c-4111-aea8-22afc5aa21d5 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:04.253187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:04.299570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0fb4b-92f7-471e-87c9-e2cbcb179b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:04.300093 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0fb4b-92f7-471e-87c9-e2cbcb179b22 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:04.300347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0fb4b-92f7-471e-87c9-e2cbcb179b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:04.300614 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0fb4b-92f7-471e-87c9-e2cbcb179b22 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:04.301284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0fb4b-92f7-471e-87c9-e2cbcb179b22 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:04.302756 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0fb4b-92f7-471e-87c9-e2cbcb179b22 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:04.322599 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 504/1020] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 23 06:01:04 2022] POST /identity/v3/services => generated 242 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 06:01:05.337572 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b89c9f-94cb-4921-9cfd-5d82b435c1a9 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:05.338093 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b89c9f-94cb-4921-9cfd-5d82b435c1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:05.338093 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b89c9f-94cb-4921-9cfd-5d82b435c1a9 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:05.340445 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 517/1021] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:05.345659 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d6569-4b6a-450c-8f28-ac1f8f5a3c2b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:05.346105 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d6569-4b6a-450c-8f28-ac1f8f5a3c2b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:05.346182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d6569-4b6a-450c-8f28-ac1f8f5a3c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:05.364942 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-dd3d6569-4b6a-450c-8f28-ac1f8f5a3c2b None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:05.366135 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-dd3d6569-4b6a-450c-8f28-ac1f8f5a3c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:05.446421 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 505/1022] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:05.712491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b89c9f-94cb-4921-9cfd-5d82b435c1a9 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:05.782825 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:05.828642 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b89c9f-94cb-4921-9cfd-5d82b435c1a9 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:05.864251 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ea32a-5bba-44dd-94b0-fb3de72c3308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:05.864888 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ea32a-5bba-44dd-94b0-fb3de72c3308 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:05.865169 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ea32a-5bba-44dd-94b0-fb3de72c3308 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:05.865443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ea32a-5bba-44dd-94b0-fb3de72c3308 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:05.866593 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ea32a-5bba-44dd-94b0-fb3de72c3308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:05.868038 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ea32a-5bba-44dd-94b0-fb3de72c3308 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:05.874554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 518/1023] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:05.876495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d6569-4b6a-450c-8f28-ac1f8f5a3c2b None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:05.972680 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:05.992857 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d6569-4b6a-450c-8f28-ac1f8f5a3c2b None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:06.031027 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11687aab-1d7d-4a12-86d9-301abfac1ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:06.031632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11687aab-1d7d-4a12-86d9-301abfac1ab1 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:06.031904 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11687aab-1d7d-4a12-86d9-301abfac1ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:06.032171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11687aab-1d7d-4a12-86d9-301abfac1ab1 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:06.032821 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11687aab-1d7d-4a12-86d9-301abfac1ab1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:06.034251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11687aab-1d7d-4a12-86d9-301abfac1ab1 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:06.057653 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 506/1024] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 06:01:05 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 06:01:06.224260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843fdae-5f2e-45d6-9fa4-6bd9c2a5a004 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:06.224446 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843fdae-5f2e-45d6-9fa4-6bd9c2a5a004 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:06.224718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843fdae-5f2e-45d6-9fa4-6bd9c2a5a004 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:06.226106 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 519/1025] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:06.234369 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbee0d9-a451-451f-8c2a-cd808646440d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:06.234805 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbee0d9-a451-451f-8c2a-cd808646440d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:06.235046 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbee0d9-a451-451f-8c2a-cd808646440d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:06.254995 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-dcbee0d9-a451-451f-8c2a-cd808646440d None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:06.256188 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-dcbee0d9-a451-451f-8c2a-cd808646440d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:06.330426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 507/1026] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:06.334064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67e1d9-c005-4c38-8f34-81addf6cef8f None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:06.334348 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67e1d9-c005-4c38-8f34-81addf6cef8f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:06.334665 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67e1d9-c005-4c38-8f34-81addf6cef8f None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:06.336124 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 520/1027] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:01:06.341286 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffcc1e-a4a1-4f35-b187-c5a27dce2097 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:06.341566 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffcc1e-a4a1-4f35-b187-c5a27dce2097 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:06.341839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffcc1e-a4a1-4f35-b187-c5a27dce2097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:06.359440 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-feffcc1e-a4a1-4f35-b187-c5a27dce2097 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:06.360616 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-feffcc1e-a4a1-4f35-b187-c5a27dce2097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:06.422851 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:06.427318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 508/1028] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:06.602215 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67e1d9-c005-4c38-8f34-81addf6cef8f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:06.695360 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffcc1e-a4a1-4f35-b187-c5a27dce2097 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:06.710599 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67e1d9-c005-4c38-8f34-81addf6cef8f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:06.743473 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:06.750538 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b094b-2afd-473d-a9c9-a135f5dd88c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:06.751172 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b094b-2afd-473d-a9c9-a135f5dd88c2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:06.751449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b094b-2afd-473d-a9c9-a135f5dd88c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:06.751719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b094b-2afd-473d-a9c9-a135f5dd88c2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:06.764795 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b094b-2afd-473d-a9c9-a135f5dd88c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-d88b094b-2afd-473d-a9c9-a135f5dd88c2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:01:06.771606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:06.772742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:06.773709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:06.773709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:06.773709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:06.773709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:06.773709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:01:06.773709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:01:06.773709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:01:06.773709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:01:06.773709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:01:06.773709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:01:06.773709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 06:01:06.773709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:06.775303 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 521/1029] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 23 06:01:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:06.779321 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b094b-2afd-473d-a9c9-a135f5dd88c2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:06.805716 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffcc1e-a4a1-4f35-b187-c5a27dce2097 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:06.843023 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916293f-652f-42f7-99d1-fbdf2a2d532d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:06.843653 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916293f-652f-42f7-99d1-fbdf2a2d532d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:06.843927 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916293f-652f-42f7-99d1-fbdf2a2d532d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:06.844193 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916293f-652f-42f7-99d1-fbdf2a2d532d None admin] PATH_INFO: `/v3/services/image` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:06.844775 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916293f-652f-42f7-99d1-fbdf2a2d532d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:06.846144 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916293f-652f-42f7-99d1-fbdf2a2d532d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-5916293f-652f-42f7-99d1-fbdf2a2d532d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:06.854884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:06.856049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:06.859080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:06.859080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:06.859080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:06.859080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:06.859080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:06.859080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:06.859080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:06.859080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:06.859080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:06.859080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:06.859080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:01:06.859080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:01:06.859080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 06:01:06.859080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:06.859080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 509/1030] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Aug 23 06:01:06 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:01:06.862452 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916293f-652f-42f7-99d1-fbdf2a2d532d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:06.907483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b094b-2afd-473d-a9c9-a135f5dd88c2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:06.934553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:06.945091 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7c74f-16c9-4806-b787-2ad980214a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:06.945706 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7c74f-16c9-4806-b787-2ad980214a40 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:06.945986 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7c74f-16c9-4806-b787-2ad980214a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:06.946256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7c74f-16c9-4806-b787-2ad980214a40 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:06.950422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7c74f-16c9-4806-b787-2ad980214a40 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:06.950422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7c74f-16c9-4806-b787-2ad980214a40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:06.950954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7c74f-16c9-4806-b787-2ad980214a40 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:06.961077 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 522/1031] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 23 06:01:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:06.966682 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7c74f-16c9-4806-b787-2ad980214a40 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:06.983550 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916293f-652f-42f7-99d1-fbdf2a2d532d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:07.021979 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7650d-61aa-4d60-b37e-323772ab3ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:07.023067 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7650d-61aa-4d60-b37e-323772ab3ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:07.023349 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7650d-61aa-4d60-b37e-323772ab3ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:07.023780 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7650d-61aa-4d60-b37e-323772ab3ce7 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:07.024418 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7650d-61aa-4d60-b37e-323772ab3ce7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:07.024740 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7650d-61aa-4d60-b37e-323772ab3ce7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:07.025709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:07.026290 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7650d-61aa-4d60-b37e-323772ab3ce7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:07.034569 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 510/1032] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:07.038933 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7650d-61aa-4d60-b37e-323772ab3ce7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:07.082151 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7c74f-16c9-4806-b787-2ad980214a40 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:07.122139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f881578-8053-45a5-98d0-626953831110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:07.122729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f881578-8053-45a5-98d0-626953831110 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:07.123014 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f881578-8053-45a5-98d0-626953831110 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:07.123284 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f881578-8053-45a5-98d0-626953831110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:07.134900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f881578-8053-45a5-98d0-626953831110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:07.136413 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f881578-8053-45a5-98d0-626953831110 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:07.145685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7650d-61aa-4d60-b37e-323772ab3ce7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-0f881578-8053-45a5-98d0-626953831110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:01:07.151491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:07.158194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:01:07.159321 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:01:07.159321 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:01:07.159321 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:07.159321 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 523/1033] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:01:07.159321 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f881578-8053-45a5-98d0-626953831110 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:07.181616 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:07.190619 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add0f58-115a-417a-ab36-0315bbe40ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:07.191343 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add0f58-115a-417a-ab36-0315bbe40ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:07.191691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add0f58-115a-417a-ab36-0315bbe40ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:07.192036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add0f58-115a-417a-ab36-0315bbe40ff9 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:07.192735 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add0f58-115a-417a-ab36-0315bbe40ff9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:07.193126 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add0f58-115a-417a-ab36-0315bbe40ff9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:07.194560 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add0f58-115a-417a-ab36-0315bbe40ff9 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:07.200894 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 511/1034] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:07.206298 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add0f58-115a-417a-ab36-0315bbe40ff9 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:07.281546 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f881578-8053-45a5-98d0-626953831110 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:07.289195 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:07.317783 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c05ae3-7772-480e-b023-47332625e00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:07.318382 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c05ae3-7772-480e-b023-47332625e00a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:07.318685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c05ae3-7772-480e-b023-47332625e00a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:07.318954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c05ae3-7772-480e-b023-47332625e00a None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:07.319209 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add0f58-115a-417a-ab36-0315bbe40ff9 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:07.319585 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c05ae3-7772-480e-b023-47332625e00a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:07.319884 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c05ae3-7772-480e-b023-47332625e00a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:07.321260 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c05ae3-7772-480e-b023-47332625e00a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:07.333942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 524/1035] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:07.339773 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c05ae3-7772-480e-b023-47332625e00a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:07.361101 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb83d87-757d-490b-9d5c-dbefa04cc409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:07.361819 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb83d87-757d-490b-9d5c-dbefa04cc409 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:07.362091 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb83d87-757d-490b-9d5c-dbefa04cc409 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:07.362357 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb83d87-757d-490b-9d5c-dbefa04cc409 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:07.363209 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb83d87-757d-490b-9d5c-dbefa04cc409 None admin] RBAC: Adding query filter params (interface=public, service_id=24a73736afaa4c61b8cb66a2c3d3dec9, region_id=RegionOne) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:07.363516 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb83d87-757d-490b-9d5c-dbefa04cc409 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:07.364980 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb83d87-757d-490b-9d5c-dbefa04cc409 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:07.375367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 512/1036] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 23 06:01:07 2022] GET /identity/v3/endpoints?service_id=24a73736afaa4c61b8cb66a2c3d3dec9&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:07.376579 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb83d87-757d-490b-9d5c-dbefa04cc409 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:07.437453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:07.452111 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c05ae3-7772-480e-b023-47332625e00a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:07.485096 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb83d87-757d-490b-9d5c-dbefa04cc409 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:07.490871 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255b634-4931-4e02-bee4-306bf3914d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:07.491707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255b634-4931-4e02-bee4-306bf3914d41 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:07.492139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255b634-4931-4e02-bee4-306bf3914d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:07.492547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255b634-4931-4e02-bee4-306bf3914d41 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:07.500262 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255b634-4931-4e02-bee4-306bf3914d41 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:07.502058 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255b634-4931-4e02-bee4-306bf3914d41 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-2255b634-4931-4e02-bee4-306bf3914d41 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:01:07.515021 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:07.515988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:07.517003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:07.517003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:07.517003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:01:07.517003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:01:07.517003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:01:07.517003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:01:07.517003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:01:07.517003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:01:07.517003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:01:07.517003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:07.517003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 525/1037] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 23 06:01:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:01:07.521967 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255b634-4931-4e02-bee4-306bf3914d41 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:07.524786 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0fdd6-48c2-44f3-8f9b-5a4e2a7a0a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:07.526084 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0fdd6-48c2-44f3-8f9b-5a4e2a7a0a0a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:07.526084 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0fdd6-48c2-44f3-8f9b-5a4e2a7a0a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:07.526084 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0fdd6-48c2-44f3-8f9b-5a4e2a7a0a0a None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:07.526566 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0fdd6-48c2-44f3-8f9b-5a4e2a7a0a0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:07.531761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0fdd6-48c2-44f3-8f9b-5a4e2a7a0a0a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:07.550799 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 513/1038] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 23 06:01:07 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 06:01:07.567799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:07.652620 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255b634-4931-4e02-bee4-306bf3914d41 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:07.690769 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aec99f-2b9c-4d5e-804c-3717ed8c020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:07.691386 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aec99f-2b9c-4d5e-804c-3717ed8c020b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:07.691664 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aec99f-2b9c-4d5e-804c-3717ed8c020b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:07.691935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aec99f-2b9c-4d5e-804c-3717ed8c020b None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:07.692671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aec99f-2b9c-4d5e-804c-3717ed8c020b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:07.692975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aec99f-2b9c-4d5e-804c-3717ed8c020b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:07.694399 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aec99f-2b9c-4d5e-804c-3717ed8c020b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:07.713649 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 526/1039] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 23 06:01:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:07.719865 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0fdd6-48c2-44f3-8f9b-5a4e2a7a0a0a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:07.823324 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0fdd6-48c2-44f3-8f9b-5a4e2a7a0a0a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:07.843897 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:07.860057 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48ab1a-198d-400c-82bd-01c2e4704866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:07.860658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48ab1a-198d-400c-82bd-01c2e4704866 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:07.860931 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48ab1a-198d-400c-82bd-01c2e4704866 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:07.861202 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48ab1a-198d-400c-82bd-01c2e4704866 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:07.942592 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48ab1a-198d-400c-82bd-01c2e4704866 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:07.943988 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48ab1a-198d-400c-82bd-01c2e4704866 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:08.058679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 514/1040] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:01:07 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:01:08.064541 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aec99f-2b9c-4d5e-804c-3717ed8c020b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:08.067595 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49546cd3-bffc-4c65-8a8e-8476622bd141 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:08.068119 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49546cd3-bffc-4c65-8a8e-8476622bd141 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:08.068624 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49546cd3-bffc-4c65-8a8e-8476622bd141 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:08.070504 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 515/1041] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:08.075983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaa00b-276a-4c30-965e-fb596565d95f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:08.076269 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaa00b-276a-4c30-965e-fb596565d95f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:08.076539 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaa00b-276a-4c30-965e-fb596565d95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:08.094669 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5deaa00b-276a-4c30-965e-fb596565d95f None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:08.095860 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5deaa00b-276a-4c30-965e-fb596565d95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:08.114056 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:08.166922 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:08.181963 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 516/1042] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:01:08.185749 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aec99f-2b9c-4d5e-804c-3717ed8c020b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:08.221410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58059581-4d45-4712-b29a-b991a74b5771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:08.222263 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58059581-4d45-4712-b29a-b991a74b5771 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:08.222729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58059581-4d45-4712-b29a-b991a74b5771 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:08.223142 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58059581-4d45-4712-b29a-b991a74b5771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:08.235512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58059581-4d45-4712-b29a-b991a74b5771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:08.236957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58059581-4d45-4712-b29a-b991a74b5771 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-58059581-4d45-4712-b29a-b991a74b5771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:01:08.253580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:08.254895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:01:08.255826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:01:08.255826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:01:08.255826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:08.255826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 527/1043] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:08.263449 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deaa00b-276a-4c30-965e-fb596565d95f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:08.364524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deaa00b-276a-4c30-965e-fb596565d95f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:08.399120 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582e5a3-499a-4d9a-a52d-75d577938926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:08.399717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582e5a3-499a-4d9a-a52d-75d577938926 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:08.399988 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582e5a3-499a-4d9a-a52d-75d577938926 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:08.400257 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582e5a3-499a-4d9a-a52d-75d577938926 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:08.400921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582e5a3-499a-4d9a-a52d-75d577938926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:08.401213 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582e5a3-499a-4d9a-a52d-75d577938926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:08.402968 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582e5a3-499a-4d9a-a52d-75d577938926 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:08.414378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 517/1044] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:08.421502 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58059581-4d45-4712-b29a-b991a74b5771 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:08.458736 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582e5a3-499a-4d9a-a52d-75d577938926 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:08.472123 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:08.534447 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58059581-4d45-4712-b29a-b991a74b5771 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:08.567183 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582e5a3-499a-4d9a-a52d-75d577938926 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:08.572830 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6713d6d-6729-44b8-87de-838c85d38d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:08.573735 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6713d6d-6729-44b8-87de-838c85d38d7f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:08.574199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6713d6d-6729-44b8-87de-838c85d38d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:08.574671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6713d6d-6729-44b8-87de-838c85d38d7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:08.579731 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:08.585167 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6713d6d-6729-44b8-87de-838c85d38d7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:08.586878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6713d6d-6729-44b8-87de-838c85d38d7f None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:08.598885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-d6713d6d-6729-44b8-87de-838c85d38d7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:01:08.598885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:08.598885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:08.598885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:08.598885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:08.598885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:08.598885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:08.598885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:08.598885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:08.598885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:08.598885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:08.598885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:08.598885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:01:08.598885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:01:08.598885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:08.600016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:08.601088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:01:08.601088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:01:08.601088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:01:08.601088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:01:08.601088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:01:08.601088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:01:08.601088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:01:08.601088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:08.601088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 528/1045] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:01:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:08.606399 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6713d6d-6729-44b8-87de-838c85d38d7f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:08.615896 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365acaa5-16f2-4576-9c48-975520c990b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:08.616559 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365acaa5-16f2-4576-9c48-975520c990b3 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:08.616841 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365acaa5-16f2-4576-9c48-975520c990b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:08.617116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365acaa5-16f2-4576-9c48-975520c990b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:08.631278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365acaa5-16f2-4576-9c48-975520c990b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-365acaa5-16f2-4576-9c48-975520c990b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:01:08.635949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:08.637713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:08.638836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:08.638836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:08.638836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:08.638836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:08.638836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:01:08.638836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:01:08.638836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:01:08.638836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:01:08.638836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:01:08.638836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:01:08.638836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:01:08.638836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:08.642762 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 518/1046] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:01:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:01:08.643663 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365acaa5-16f2-4576-9c48-975520c990b3 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:08.719271 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:08.725152 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6713d6d-6729-44b8-87de-838c85d38d7f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:08.752738 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365acaa5-16f2-4576-9c48-975520c990b3 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:08.763755 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bff37-6fae-4654-8c08-2851ac6ef03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:08.764246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bff37-6fae-4654-8c08-2851ac6ef03b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:08.764524 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bff37-6fae-4654-8c08-2851ac6ef03b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:08.764810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bff37-6fae-4654-8c08-2851ac6ef03b None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:08.765502 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bff37-6fae-4654-8c08-2851ac6ef03b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:08.765806 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bff37-6fae-4654-8c08-2851ac6ef03b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:08.767561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bff37-6fae-4654-8c08-2851ac6ef03b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:08.780670 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 529/1047] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:01:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:08.786588 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bff37-6fae-4654-8c08-2851ac6ef03b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:08.792666 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daddbc2-5975-40fd-81af-49ddb43c7b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:08.793274 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daddbc2-5975-40fd-81af-49ddb43c7b96 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:08.793550 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daddbc2-5975-40fd-81af-49ddb43c7b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:08.793820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daddbc2-5975-40fd-81af-49ddb43c7b96 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:08.794481 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daddbc2-5975-40fd-81af-49ddb43c7b96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:08.794808 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daddbc2-5975-40fd-81af-49ddb43c7b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:08.796177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daddbc2-5975-40fd-81af-49ddb43c7b96 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:08.805380 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 519/1048] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:01:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:08.811848 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daddbc2-5975-40fd-81af-49ddb43c7b96 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:08.849007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:08.893428 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bff37-6fae-4654-8c08-2851ac6ef03b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:08.937951 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b0c42-2226-462a-9e25-bebfa0cc6f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:08.938591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b0c42-2226-462a-9e25-bebfa0cc6f83 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:08.938874 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b0c42-2226-462a-9e25-bebfa0cc6f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:08.939140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b0c42-2226-462a-9e25-bebfa0cc6f83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:08.939891 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b0c42-2226-462a-9e25-bebfa0cc6f83 None admin] RBAC: Adding query filter params (role.id=aa12d72368054e13a59fa2a814d51afd, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=91267a049343452eb208205ee3ea27c4) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:08.940194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b0c42-2226-462a-9e25-bebfa0cc6f83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:08.941617 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b0c42-2226-462a-9e25-bebfa0cc6f83 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:08.944431 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daddbc2-5975-40fd-81af-49ddb43c7b96 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:08.947157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 530/1049] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:01:08 2022] GET /identity/v3/role_assignments?user.id=91267a049343452eb208205ee3ea27c4&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=aa12d72368054e13a59fa2a814d51afd => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:08.989119 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:08.995525 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50008efa-ec97-4069-bc14-5a74610beb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:08.996463 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50008efa-ec97-4069-bc14-5a74610beb70 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:08.996888 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50008efa-ec97-4069-bc14-5a74610beb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:08.997170 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50008efa-ec97-4069-bc14-5a74610beb70 None admin] PATH_INFO: `/v3/users/20b1751bc12b4dca85ef790bcb217c2e` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:09.014153 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50008efa-ec97-4069-bc14-5a74610beb70 None admin] RBAC: Authorizing `identity:get_user(user_id=20b1751bc12b4dca85ef790bcb217c2e)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:09.015680 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50008efa-ec97-4069-bc14-5a74610beb70 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:09.017649 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 520/1050] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 06:01:08 2022] GET /identity/v3/users/20b1751bc12b4dca85ef790bcb217c2e => generated 285 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:09.023581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b0c42-2226-462a-9e25-bebfa0cc6f83 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:09.140615 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b0c42-2226-462a-9e25-bebfa0cc6f83 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:09.160870 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:09.177196 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65b357-e9c6-4308-bdad-6857dc6cc30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:09.177814 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65b357-e9c6-4308-bdad-6857dc6cc30d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:09.178099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65b357-e9c6-4308-bdad-6857dc6cc30d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:09.178383 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65b357-e9c6-4308-bdad-6857dc6cc30d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:09.274343 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65b357-e9c6-4308-bdad-6857dc6cc30d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:09.276137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65b357-e9c6-4308-bdad-6857dc6cc30d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:09.382488 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 531/1051] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:01:09 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:01:09.388456 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50008efa-ec97-4069-bc14-5a74610beb70 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:09.495995 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50008efa-ec97-4069-bc14-5a74610beb70 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:09.515136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655b815-9a54-4937-8fbe-ad1e6a5da7f4 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:09.515436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655b815-9a54-4937-8fbe-ad1e6a5da7f4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:09.515685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655b815-9a54-4937-8fbe-ad1e6a5da7f4 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:09.517184 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 532/1052] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:01:09.521965 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc00118-0e08-4867-8e7c-5106206b5303 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:09.522244 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc00118-0e08-4867-8e7c-5106206b5303 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:09.522530 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc00118-0e08-4867-8e7c-5106206b5303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:09.532445 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06976520-3fd0-4297-8848-2ceb77eb4f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:09.533010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06976520-3fd0-4297-8848-2ceb77eb4f0d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:09.533288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06976520-3fd0-4297-8848-2ceb77eb4f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:09.533560 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06976520-3fd0-4297-8848-2ceb77eb4f0d None admin] PATH_INFO: `/v3/projects/08405881684e4a9fa222268105021f1a` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:09.534726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06976520-3fd0-4297-8848-2ceb77eb4f0d None admin] RBAC: Authorizing `identity:get_project(project_id=08405881684e4a9fa222268105021f1a)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:09.536154 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06976520-3fd0-4297-8848-2ceb77eb4f0d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:09.543290 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 521/1053] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 06:01:09 2022] GET /identity/v3/projects/08405881684e4a9fa222268105021f1a => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:09.545097 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06976520-3fd0-4297-8848-2ceb77eb4f0d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:09.548774 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ebc00118-0e08-4867-8e7c-5106206b5303 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:09.550625 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ebc00118-0e08-4867-8e7c-5106206b5303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:09.571910 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:09.611200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:09.631765 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 533/1054] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:01:09.656529 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06976520-3fd0-4297-8848-2ceb77eb4f0d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:09.692981 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c05f2-5888-43bf-820a-0031ee59c031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:09.693584 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c05f2-5888-43bf-820a-0031ee59c031 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:09.693858 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c05f2-5888-43bf-820a-0031ee59c031 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:09.694122 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c05f2-5888-43bf-820a-0031ee59c031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:09.694876 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c05f2-5888-43bf-820a-0031ee59c031 None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=08405881684e4a9fa222268105021f1a, user.id=20b1751bc12b4dca85ef790bcb217c2e) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:09.695171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c05f2-5888-43bf-820a-0031ee59c031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:09.696597 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c05f2-5888-43bf-820a-0031ee59c031 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:09.710588 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 522/1055] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:01:09 2022] GET /identity/v3/role_assignments?user.id=20b1751bc12b4dca85ef790bcb217c2e&scope.project.id=08405881684e4a9fa222268105021f1a&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:09.904704 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc00118-0e08-4867-8e7c-5106206b5303 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:10.015823 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc00118-0e08-4867-8e7c-5106206b5303 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:10.054318 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b2816-5371-47e7-afe8-d4183c6f16a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:10.054973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b2816-5371-47e7-afe8-d4183c6f16a8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:10.055255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b2816-5371-47e7-afe8-d4183c6f16a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:10.055526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b2816-5371-47e7-afe8-d4183c6f16a8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:10.056105 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b2816-5371-47e7-afe8-d4183c6f16a8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:10.057468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b2816-5371-47e7-afe8-d4183c6f16a8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-ae0b2816-5371-47e7-afe8-d4183c6f16a8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:10.066149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:10.067219 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:10.068067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:10.068067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:10.068067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:10.068067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:10.068067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:10.068067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:10.068067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:10.068067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:10.068067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:10.068067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:10.068067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:01:10.068067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:01:10.068067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 06:01:10.068067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:10.069453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 534/1056] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Aug 23 06:01:09 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:10.073161 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c05f2-5888-43bf-820a-0031ee59c031 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:10.180123 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c05f2-5888-43bf-820a-0031ee59c031 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:10.204362 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:10.218346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633dccc0-b187-40ee-ac50-b0c0b6f19b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:10.218984 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633dccc0-b187-40ee-ac50-b0c0b6f19b99 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:10.219264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633dccc0-b187-40ee-ac50-b0c0b6f19b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:10.219556 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633dccc0-b187-40ee-ac50-b0c0b6f19b99 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:10.220225 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633dccc0-b187-40ee-ac50-b0c0b6f19b99 None admin] RBAC: Adding query filter params (name=image) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:10.220546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633dccc0-b187-40ee-ac50-b0c0b6f19b99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:10.221943 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633dccc0-b187-40ee-ac50-b0c0b6f19b99 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:10.227151 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 523/1057] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:10.232870 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b2816-5371-47e7-afe8-d4183c6f16a8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:10.252027 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:10.341550 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b2816-5371-47e7-afe8-d4183c6f16a8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:10.376434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d1566-4a69-4d1b-bde4-a1805ff7fc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:10.377038 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d1566-4a69-4d1b-bde4-a1805ff7fc93 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:10.377315 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d1566-4a69-4d1b-bde4-a1805ff7fc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:10.377596 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d1566-4a69-4d1b-bde4-a1805ff7fc93 None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:10.378206 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d1566-4a69-4d1b-bde4-a1805ff7fc93 None admin] RBAC: Adding query filter params (type=image) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:10.378503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d1566-4a69-4d1b-bde4-a1805ff7fc93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:10.379887 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d1566-4a69-4d1b-bde4-a1805ff7fc93 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:10.390436 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 535/1058] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:10.391818 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633dccc0-b187-40ee-ac50-b0c0b6f19b99 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:10.496107 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633dccc0-b187-40ee-ac50-b0c0b6f19b99 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:10.531426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2af44e-f880-4173-a6d5-a9bab0b95ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:10.532032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2af44e-f880-4173-a6d5-a9bab0b95ceb None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:10.532307 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2af44e-f880-4173-a6d5-a9bab0b95ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:10.532672 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2af44e-f880-4173-a6d5-a9bab0b95ceb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:10.533227 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2af44e-f880-4173-a6d5-a9bab0b95ceb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:10.534619 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2af44e-f880-4173-a6d5-a9bab0b95ceb None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:10.562561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 524/1059] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 23 06:01:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 06:01:10.836907 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7909ca6-cd80-4480-bdef-e212031b4b5c None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:10.837325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7909ca6-cd80-4480-bdef-e212031b4b5c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:10.837440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7909ca6-cd80-4480-bdef-e212031b4b5c None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:10.838853 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 536/1060] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:10.844151 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028f959-77c7-400a-9f82-4a4655df1dba None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:10.844428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028f959-77c7-400a-9f82-4a4655df1dba None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:10.844693 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028f959-77c7-400a-9f82-4a4655df1dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:10.863189 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1028f959-77c7-400a-9f82-4a4655df1dba None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:10.864359 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1028f959-77c7-400a-9f82-4a4655df1dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:10.902874 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:10.950017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 525/1061] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:01:11.220935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7909ca6-cd80-4480-bdef-e212031b4b5c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:11.308681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:11.335294 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7909ca6-cd80-4480-bdef-e212031b4b5c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:11.371250 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38104463-8e68-4f02-bd67-49807b392102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:11.371880 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38104463-8e68-4f02-bd67-49807b392102 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:11.372159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38104463-8e68-4f02-bd67-49807b392102 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:11.372428 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38104463-8e68-4f02-bd67-49807b392102 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:11.383597 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38104463-8e68-4f02-bd67-49807b392102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-38104463-8e68-4f02-bd67-49807b392102 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:01:11.388834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:11.389789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:11.390668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:11.390668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:11.390668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:11.390668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:11.390668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:01:11.390668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:01:11.390668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:01:11.390668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:01:11.390668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:01:11.390668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:01:11.390668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 06:01:11.390668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:11.390668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 537/1062] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 23 06:01:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:11.396526 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028f959-77c7-400a-9f82-4a4655df1dba None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:11.500245 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028f959-77c7-400a-9f82-4a4655df1dba None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:11.534891 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b731649-7be7-4701-9c91-4deda41e5222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:11.535502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b731649-7be7-4701-9c91-4deda41e5222 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:11.535780 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b731649-7be7-4701-9c91-4deda41e5222 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:11.536047 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b731649-7be7-4701-9c91-4deda41e5222 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:11.536707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b731649-7be7-4701-9c91-4deda41e5222 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:11.537004 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b731649-7be7-4701-9c91-4deda41e5222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:11.538367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b731649-7be7-4701-9c91-4deda41e5222 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:11.547496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 526/1063] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 23 06:01:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:11.553157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38104463-8e68-4f02-bd67-49807b392102 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:11.644508 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f875e-c4bc-4abd-9484-72e06c1fc470 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:11.644832 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f875e-c4bc-4abd-9484-72e06c1fc470 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:11.645062 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f875e-c4bc-4abd-9484-72e06c1fc470 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:11.646587 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 527/1064] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:11.651538 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449edf83-65d0-4e77-98f3-11b93bae6aec None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:11.651846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449edf83-65d0-4e77-98f3-11b93bae6aec None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:11.652121 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449edf83-65d0-4e77-98f3-11b93bae6aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:11.659646 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38104463-8e68-4f02-bd67-49807b392102 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:11.671274 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-449edf83-65d0-4e77-98f3-11b93bae6aec None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:11.672498 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-449edf83-65d0-4e77-98f3-11b93bae6aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:11.697726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:11.699678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280efe6c-95a0-4009-9b94-f81c1eb0e9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:11.700378 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280efe6c-95a0-4009-9b94-f81c1eb0e9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:11.700657 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280efe6c-95a0-4009-9b94-f81c1eb0e9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:11.700929 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280efe6c-95a0-4009-9b94-f81c1eb0e9f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:11.708156 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:11.712489 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280efe6c-95a0-4009-9b94-f81c1eb0e9f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:11.713930 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280efe6c-95a0-4009-9b94-f81c1eb0e9f9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-280efe6c-95a0-4009-9b94-f81c1eb0e9f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:01:11.729472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:11.731236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:01:11.732167 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:01:11.732167 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:01:11.732167 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:11.732167 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 538/1065] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:11.738083 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280efe6c-95a0-4009-9b94-f81c1eb0e9f9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:11.752304 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 528/1066] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:01:11.840360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280efe6c-95a0-4009-9b94-f81c1eb0e9f9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:11.874851 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9aecc-a885-49ec-b333-41125fcffc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:11.875451 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9aecc-a885-49ec-b333-41125fcffc89 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:11.875728 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9aecc-a885-49ec-b333-41125fcffc89 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:11.875994 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9aecc-a885-49ec-b333-41125fcffc89 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:11.876631 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9aecc-a885-49ec-b333-41125fcffc89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:11.876930 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9aecc-a885-49ec-b333-41125fcffc89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:11.878278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9aecc-a885-49ec-b333-41125fcffc89 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:11.895247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 539/1067] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:11.897118 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449edf83-65d0-4e77-98f3-11b93bae6aec None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:12.010827 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449edf83-65d0-4e77-98f3-11b93bae6aec None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:12.016515 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9aecc-a885-49ec-b333-41125fcffc89 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:12.054441 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f46a98-ae27-4c0b-96da-da504cf71976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:12.055120 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f46a98-ae27-4c0b-96da-da504cf71976 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:12.055398 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f46a98-ae27-4c0b-96da-da504cf71976 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:12.055671 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f46a98-ae27-4c0b-96da-da504cf71976 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:12.065541 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f46a98-ae27-4c0b-96da-da504cf71976 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:12.067046 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f46a98-ae27-4c0b-96da-da504cf71976 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:12.071029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-d1f46a98-ae27-4c0b-96da-da504cf71976 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:01:12.084515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:12.085510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:12.086459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:12.086459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:12.086459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:01:12.086459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:01:12.086459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:01:12.086459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:01:12.086459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:01:12.086459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:01:12.086459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:01:12.086459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:12.086459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 529/1068] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 23 06:01:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:12.091885 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f46a98-ae27-4c0b-96da-da504cf71976 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:12.115286 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:12.136781 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9aecc-a885-49ec-b333-41125fcffc89 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:12.175796 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb8334-8579-4e18-a007-5288bbf45d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:12.176558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb8334-8579-4e18-a007-5288bbf45d1a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:12.176841 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb8334-8579-4e18-a007-5288bbf45d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:12.177109 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb8334-8579-4e18-a007-5288bbf45d1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:12.188453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb8334-8579-4e18-a007-5288bbf45d1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-8deb8334-8579-4e18-a007-5288bbf45d1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:01:12.193607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:12.194712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:12.195579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:12.195579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:12.195579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:12.195579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:12.195579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:01:12.195579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:01:12.195579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:01:12.195579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:01:12.195579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:01:12.195579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:01:12.195579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:01:12.195579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:12.195579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 540/1069] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:01:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:01:12.200724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f46a98-ae27-4c0b-96da-da504cf71976 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:12.202241 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb8334-8579-4e18-a007-5288bbf45d1a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:12.237847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e4939-94e4-4770-a5b4-d8fbe47d39b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:12.238451 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e4939-94e4-4770-a5b4-d8fbe47d39b7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:12.238754 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e4939-94e4-4770-a5b4-d8fbe47d39b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:12.239020 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e4939-94e4-4770-a5b4-d8fbe47d39b7 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:12.239756 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e4939-94e4-4770-a5b4-d8fbe47d39b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:12.240057 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e4939-94e4-4770-a5b4-d8fbe47d39b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:12.241495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e4939-94e4-4770-a5b4-d8fbe47d39b7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:12.262617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 530/1070] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 23 06:01:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:12.268171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e4939-94e4-4770-a5b4-d8fbe47d39b7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:12.326305 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb8334-8579-4e18-a007-5288bbf45d1a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:12.338484 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:12.359456 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:12.368048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc515b-f5ea-43f6-8b96-023e24493802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:12.369046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc515b-f5ea-43f6-8b96-023e24493802 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:12.369415 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc515b-f5ea-43f6-8b96-023e24493802 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:12.369681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc515b-f5ea-43f6-8b96-023e24493802 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:12.370338 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc515b-f5ea-43f6-8b96-023e24493802 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:12.370697 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc515b-f5ea-43f6-8b96-023e24493802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:12.372060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc515b-f5ea-43f6-8b96-023e24493802 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:12.380631 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 541/1071] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:01:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:12.386704 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e4939-94e4-4770-a5b4-d8fbe47d39b7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:12.387058 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc515b-f5ea-43f6-8b96-023e24493802 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:12.424518 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bee0a-0430-4843-bb35-b4b83f1a6f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:12.425142 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bee0a-0430-4843-bb35-b4b83f1a6f8e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:12.425425 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bee0a-0430-4843-bb35-b4b83f1a6f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:12.425698 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bee0a-0430-4843-bb35-b4b83f1a6f8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:12.437058 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bee0a-0430-4843-bb35-b4b83f1a6f8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:12.438508 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bee0a-0430-4843-bb35-b4b83f1a6f8e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-683bee0a-0430-4843-bb35-b4b83f1a6f8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:01:12.453640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:12.454846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:01:12.455912 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:01:12.455912 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:01:12.455912 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:12.459487 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 531/1072] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:01:12.460340 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bee0a-0430-4843-bb35-b4b83f1a6f8e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:12.502864 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc515b-f5ea-43f6-8b96-023e24493802 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:12.539192 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03465fdc-9da7-4e00-922d-c12a157d0d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:12.539765 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03465fdc-9da7-4e00-922d-c12a157d0d1c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:12.540073 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03465fdc-9da7-4e00-922d-c12a157d0d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:12.540365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03465fdc-9da7-4e00-922d-c12a157d0d1c None admin] PATH_INFO: `/v3/users/20b1751bc12b4dca85ef790bcb217c2e` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:12.541573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03465fdc-9da7-4e00-922d-c12a157d0d1c None admin] RBAC: Authorizing `identity:get_user(user_id=20b1751bc12b4dca85ef790bcb217c2e)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:12.543220 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03465fdc-9da7-4e00-922d-c12a157d0d1c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:12.544820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 542/1073] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 06:01:12 2022] GET /identity/v3/users/20b1751bc12b4dca85ef790bcb217c2e => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:12.545947 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54611d76-43b5-45ae-bdcc-6af61bd87e5a None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:12.546230 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54611d76-43b5-45ae-bdcc-6af61bd87e5a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:12.546500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54611d76-43b5-45ae-bdcc-6af61bd87e5a None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:12.547732 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 543/1074] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:12.550900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54611d76-43b5-45ae-bdcc-6af61bd87e5a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:12.572237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bee0a-0430-4843-bb35-b4b83f1a6f8e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:12.595221 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:12.602235 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:12.609341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73dc28-e0cc-4bef-9861-a181ed7d722e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:12.609949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73dc28-e0cc-4bef-9861-a181ed7d722e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:12.610233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73dc28-e0cc-4bef-9861-a181ed7d722e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:12.610495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73dc28-e0cc-4bef-9861-a181ed7d722e None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:12.611253 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73dc28-e0cc-4bef-9861-a181ed7d722e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:12.611553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73dc28-e0cc-4bef-9861-a181ed7d722e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:12.612921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73dc28-e0cc-4bef-9861-a181ed7d722e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:12.626619 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 532/1075] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:12.627663 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c6482-73a8-45c4-b9b1-43413c6da88f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:12.627947 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c6482-73a8-45c4-b9b1-43413c6da88f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:12.628214 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c6482-73a8-45c4-b9b1-43413c6da88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:12.646289 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-d06c6482-73a8-45c4-b9b1-43413c6da88f None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:12.647521 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d06c6482-73a8-45c4-b9b1-43413c6da88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:12.657685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54611d76-43b5-45ae-bdcc-6af61bd87e5a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:12.692664 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85498e7-cb37-4f43-aec0-265a03a9634d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:12.693293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85498e7-cb37-4f43-aec0-265a03a9634d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:12.693573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85498e7-cb37-4f43-aec0-265a03a9634d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:12.693845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85498e7-cb37-4f43-aec0-265a03a9634d None admin] PATH_INFO: `/v3/projects/08405881684e4a9fa222268105021f1a` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:12.694983 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85498e7-cb37-4f43-aec0-265a03a9634d None admin] RBAC: Authorizing `identity:get_project(project_id=08405881684e4a9fa222268105021f1a)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:12.696414 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85498e7-cb37-4f43-aec0-265a03a9634d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:12.698063 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 544/1076] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 06:01:12 2022] GET /identity/v3/projects/08405881684e4a9fa222268105021f1a => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:12.703444 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85498e7-cb37-4f43-aec0-265a03a9634d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:12.719651 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 533/1077] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:12.723430 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c6482-73a8-45c4-b9b1-43413c6da88f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:12.815240 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85498e7-cb37-4f43-aec0-265a03a9634d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:12.826550 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:12.835076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c6482-73a8-45c4-b9b1-43413c6da88f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:12.857473 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a42cc8-fc89-48bf-94af-6e356c4fac6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:12.857946 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:12.858218 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a42cc8-fc89-48bf-94af-6e356c4fac6c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:12.858493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a42cc8-fc89-48bf-94af-6e356c4fac6c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:12.858785 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a42cc8-fc89-48bf-94af-6e356c4fac6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:12.865775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a42cc8-fc89-48bf-94af-6e356c4fac6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:12.867317 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a42cc8-fc89-48bf-94af-6e356c4fac6c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:12.875176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c6c9a-781d-4c6c-ab59-c5b31f52d34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:12.875828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c6c9a-781d-4c6c-ab59-c5b31f52d34e None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:12.876098 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c6c9a-781d-4c6c-ab59-c5b31f52d34e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:12.876363 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c6c9a-781d-4c6c-ab59-c5b31f52d34e None admin] PATH_INFO: `/v3/projects/08405881684e4a9fa222268105021f1a/users/20b1751bc12b4dca85ef790bcb217c2e/roles/f6acb6fdca5a44de9ba00b2b0d4d3202` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:12.878241 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c6c9a-781d-4c6c-ab59-c5b31f52d34e None admin] RBAC: Authorizing `identity:create_grant(project_id=08405881684e4a9fa222268105021f1a, user_id=20b1751bc12b4dca85ef790bcb217c2e, role_id=f6acb6fdca5a44de9ba00b2b0d4d3202)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:12.879708 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c6c9a-781d-4c6c-ab59-c5b31f52d34e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:12.880396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-25a42cc8-fc89-48bf-94af-6e356c4fac6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:01:12.880396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:12.880396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:12.880396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:12.880396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:12.880396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:12.880396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:12.880396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:12.880396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:12.880396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:12.880396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:12.880396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:12.880396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:01:12.880396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:01:12.880396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:12.881232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:12.882139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:01:12.882139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:01:12.882139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:01:12.882139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:01:12.882139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:01:12.882139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:01:12.882139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:01:12.882139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:12.882139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 545/1078] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:01:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:12.888726 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a42cc8-fc89-48bf-94af-6e356c4fac6c None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:12.896610 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 534/1079] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:01:12 2022] PUT /identity/v3/projects/08405881684e4a9fa222268105021f1a/users/20b1751bc12b4dca85ef790bcb217c2e/roles/f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:01:12.994538 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c6c9a-781d-4c6c-ab59-c5b31f52d34e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:13.017386 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a42cc8-fc89-48bf-94af-6e356c4fac6c None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:13.061813 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6aac0-3c63-4a57-a780-02aed742f705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:13.062425 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6aac0-3c63-4a57-a780-02aed742f705 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:13.062734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6aac0-3c63-4a57-a780-02aed742f705 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:13.063002 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6aac0-3c63-4a57-a780-02aed742f705 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:13.063699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6aac0-3c63-4a57-a780-02aed742f705 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:13.064005 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6aac0-3c63-4a57-a780-02aed742f705 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:13.065487 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6aac0-3c63-4a57-a780-02aed742f705 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:13.079181 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 546/1080] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:01:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:13.086426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6aac0-3c63-4a57-a780-02aed742f705 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:13.109454 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c6c9a-781d-4c6c-ab59-c5b31f52d34e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:13.116815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:13.147611 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f991d-2ac5-4c65-ac15-9f305b2a09c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:13.148013 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f991d-2ac5-4c65-ac15-9f305b2a09c1 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:13.148289 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f991d-2ac5-4c65-ac15-9f305b2a09c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:13.148553 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f991d-2ac5-4c65-ac15-9f305b2a09c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:13.197684 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6aac0-3c63-4a57-a780-02aed742f705 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:13.205183 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:13.228174 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f991d-2ac5-4c65-ac15-9f305b2a09c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:13.229670 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f991d-2ac5-4c65-ac15-9f305b2a09c1 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:13.236352 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea3c7d-c205-4a2b-86db-663b398fbdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:13.237017 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea3c7d-c205-4a2b-86db-663b398fbdae None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:13.237291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea3c7d-c205-4a2b-86db-663b398fbdae None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:13.237558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea3c7d-c205-4a2b-86db-663b398fbdae None admin] PATH_INFO: `/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/91267a049343452eb208205ee3ea27c4/roles/aa12d72368054e13a59fa2a814d51afd` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:13.239986 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea3c7d-c205-4a2b-86db-663b398fbdae None admin] RBAC: Authorizing `identity:create_grant(project_id=b0b43dead1664475b9dd52113e08d7ed, user_id=91267a049343452eb208205ee3ea27c4, role_id=aa12d72368054e13a59fa2a814d51afd)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:13.241406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea3c7d-c205-4a2b-86db-663b398fbdae None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:13.262602 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 547/1081] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:01:13 2022] PUT /identity/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/91267a049343452eb208205ee3ea27c4/roles/aa12d72368054e13a59fa2a814d51afd => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 06:01:13.358656 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 535/1082] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:01:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 06:01:13.364407 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea3c7d-c205-4a2b-86db-663b398fbdae None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:13.469913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:13.485730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea3c7d-c205-4a2b-86db-663b398fbdae None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:13.520967 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab6028-669d-4305-bad3-b686c19ff1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:13.521593 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab6028-669d-4305-bad3-b686c19ff1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:13.521864 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab6028-669d-4305-bad3-b686c19ff1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:13.522165 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab6028-669d-4305-bad3-b686c19ff1c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:13.532235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab6028-669d-4305-bad3-b686c19ff1c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:13.533732 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab6028-669d-4305-bad3-b686c19ff1c7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-98ab6028-669d-4305-bad3-b686c19ff1c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:01:13.548812 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:13.549778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:01:13.551099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:01:13.551099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:01:13.551099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:13.551099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 548/1083] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:01:13.556564 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f991d-2ac5-4c65-ac15-9f305b2a09c1 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:13.649996 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:13.671560 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f991d-2ac5-4c65-ac15-9f305b2a09c1 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:13.705803 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69485a-94c6-4778-90e4-1ab0315f0004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:13.706405 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69485a-94c6-4778-90e4-1ab0315f0004 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:13.706711 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69485a-94c6-4778-90e4-1ab0315f0004 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:13.706979 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69485a-94c6-4778-90e4-1ab0315f0004 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:13.707605 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69485a-94c6-4778-90e4-1ab0315f0004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:13.707901 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69485a-94c6-4778-90e4-1ab0315f0004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:13.709256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69485a-94c6-4778-90e4-1ab0315f0004 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:13.726555 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 536/1084] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:14.859512 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b0344-8f65-4967-85ab-5eb475d4abdc None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:14.860060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b0344-8f65-4967-85ab-5eb475d4abdc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:14.860060 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b0344-8f65-4967-85ab-5eb475d4abdc None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:14.861386 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 549/1085] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:14.866733 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c7bd3-cff0-47ad-9604-23c06a977fe3 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:14.867154 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c7bd3-cff0-47ad-9604-23c06a977fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:14.867287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c7bd3-cff0-47ad-9604-23c06a977fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:14.886407 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-405c7bd3-cff0-47ad-9604-23c06a977fe3 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:14.887863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-405c7bd3-cff0-47ad-9604-23c06a977fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:14.963690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 537/1086] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:15.210466 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cf437-4a30-4944-b8be-f0e2b5f73d52 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:15.210789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cf437-4a30-4944-b8be-f0e2b5f73d52 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:15.210976 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cf437-4a30-4944-b8be-f0e2b5f73d52 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:15.212429 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 550/1087] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:15.217684 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a26d2d-e9a5-4f99-8988-dd49108c9e3c None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:15.217965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a26d2d-e9a5-4f99-8988-dd49108c9e3c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:15.218242 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a26d2d-e9a5-4f99-8988-dd49108c9e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:15.230982 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129cf437-4a30-4944-b8be-f0e2b5f73d52 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:15.237859 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-98a26d2d-e9a5-4f99-8988-dd49108c9e3c None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:15.239091 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-98a26d2d-e9a5-4f99-8988-dd49108c9e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:15.304870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:15.312886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 538/1088] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:15.346580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129cf437-4a30-4944-b8be-f0e2b5f73d52 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:15.386469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f0e1c-e1b7-4b09-9002-fec6c29e9cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:15.387111 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f0e1c-e1b7-4b09-9002-fec6c29e9cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:15.387387 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f0e1c-e1b7-4b09-9002-fec6c29e9cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:15.387656 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f0e1c-e1b7-4b09-9002-fec6c29e9cd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:15.395064 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:15.398862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f0e1c-e1b7-4b09-9002-fec6c29e9cd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-748f0e1c-e1b7-4b09-9002-fec6c29e9cd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:01:15.403939 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:15.405376 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:15.406273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:15.406273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:15.406273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:15.406273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:15.406273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:01:15.406273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:01:15.406273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:01:15.406273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:01:15.406273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:01:15.406273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:01:15.406273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:01:15.406273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:15.406273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 551/1089] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:01:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:01:15.411090 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a26d2d-e9a5-4f99-8988-dd49108c9e3c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:15.511769 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a26d2d-e9a5-4f99-8988-dd49108c9e3c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:15.546849 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674594a-d711-49d7-baeb-234d2cf2778e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:15.547444 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674594a-d711-49d7-baeb-234d2cf2778e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:15.547716 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674594a-d711-49d7-baeb-234d2cf2778e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:15.547981 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674594a-d711-49d7-baeb-234d2cf2778e None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:15.548628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674594a-d711-49d7-baeb-234d2cf2778e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:15.548921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674594a-d711-49d7-baeb-234d2cf2778e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:15.550278 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674594a-d711-49d7-baeb-234d2cf2778e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:15.563266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 539/1090] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:01:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:15.564421 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f0e1c-e1b7-4b09-9002-fec6c29e9cd6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:15.594766 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674594a-d711-49d7-baeb-234d2cf2778e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:15.659909 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:15.666123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f0e1c-e1b7-4b09-9002-fec6c29e9cd6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:15.700529 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674594a-d711-49d7-baeb-234d2cf2778e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:15.701696 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb3925-df31-4030-b2dd-50327013adf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:15.702321 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb3925-df31-4030-b2dd-50327013adf9 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:15.702629 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb3925-df31-4030-b2dd-50327013adf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:15.702900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb3925-df31-4030-b2dd-50327013adf9 None admin] PATH_INFO: `/v3/users/20b1751bc12b4dca85ef790bcb217c2e` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:15.704013 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb3925-df31-4030-b2dd-50327013adf9 None admin] RBAC: Authorizing `identity:get_user(user_id=20b1751bc12b4dca85ef790bcb217c2e)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:15.705430 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb3925-df31-4030-b2dd-50327013adf9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:15.707041 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 552/1091] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 06:01:15 2022] GET /identity/v3/users/20b1751bc12b4dca85ef790bcb217c2e => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:15.712986 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb3925-df31-4030-b2dd-50327013adf9 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:15.734270 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1fa0c-9e8c-4c2b-b105-05c349cecf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:15.734922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1fa0c-9e8c-4c2b-b105-05c349cecf57 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:15.735194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1fa0c-9e8c-4c2b-b105-05c349cecf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:15.735458 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1fa0c-9e8c-4c2b-b105-05c349cecf57 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:15.746814 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1fa0c-9e8c-4c2b-b105-05c349cecf57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-01c1fa0c-9e8c-4c2b-b105-05c349cecf57 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:01:15.751453 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:15.752415 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:15.753261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:15.753261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:15.753261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:15.753261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:15.753261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:01:15.753261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:01:15.753261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:01:15.753261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:01:15.753261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:01:15.753261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:01:15.753261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 06:01:15.753261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:15.753261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 540/1092] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Aug 23 06:01:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:01:15.758451 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1fa0c-9e8c-4c2b-b105-05c349cecf57 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:15.783271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:15.818765 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb3925-df31-4030-b2dd-50327013adf9 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:15.853073 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b3388-48d8-4e1e-b85f-8511d4555ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:15.853679 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b3388-48d8-4e1e-b85f-8511d4555ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:15.853960 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b3388-48d8-4e1e-b85f-8511d4555ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:15.854231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b3388-48d8-4e1e-b85f-8511d4555ff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:15.869302 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1fa0c-9e8c-4c2b-b105-05c349cecf57 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:15.892721 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:15.910885 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02ec28-7b56-4114-b255-5d5b06632fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:15.911494 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02ec28-7b56-4114-b255-5d5b06632fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:15.911774 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02ec28-7b56-4114-b255-5d5b06632fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:15.912094 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02ec28-7b56-4114-b255-5d5b06632fb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:15.912769 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02ec28-7b56-4114-b255-5d5b06632fb2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:15.913069 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02ec28-7b56-4114-b255-5d5b06632fb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:15.914433 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02ec28-7b56-4114-b255-5d5b06632fb2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:15.926545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 541/1093] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 23 06:01:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:15.926723 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b3388-48d8-4e1e-b85f-8511d4555ff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:15.926723 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b3388-48d8-4e1e-b85f-8511d4555ff6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:15.929070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02ec28-7b56-4114-b255-5d5b06632fb2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:15.997388 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:16.031322 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 553/1094] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:01:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 06:01:16.031846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02ec28-7b56-4114-b255-5d5b06632fb2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:16.037197 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b3388-48d8-4e1e-b85f-8511d4555ff6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:16.070877 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7be02c-d1be-400f-b7ec-781639fb58ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:16.071502 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7be02c-d1be-400f-b7ec-781639fb58ed None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:16.071784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7be02c-d1be-400f-b7ec-781639fb58ed None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:16.072055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7be02c-d1be-400f-b7ec-781639fb58ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:16.082760 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7be02c-d1be-400f-b7ec-781639fb58ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:16.084261 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7be02c-d1be-400f-b7ec-781639fb58ed None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-8b7be02c-d1be-400f-b7ec-781639fb58ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:01:16.100845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:16.102847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:01:16.105270 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:01:16.105270 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:01:16.105270 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:16.105270 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 542/1095] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:16.108458 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7be02c-d1be-400f-b7ec-781639fb58ed None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:16.155102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:16.158841 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b3388-48d8-4e1e-b85f-8511d4555ff6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:16.195556 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb38fa1-a0b3-490d-b4ae-e61dafb85801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:16.196185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb38fa1-a0b3-490d-b4ae-e61dafb85801 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:16.196464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb38fa1-a0b3-490d-b4ae-e61dafb85801 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:16.196738 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb38fa1-a0b3-490d-b4ae-e61dafb85801 None admin] PATH_INFO: `/v3/projects/08405881684e4a9fa222268105021f1a` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:16.197852 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb38fa1-a0b3-490d-b4ae-e61dafb85801 None admin] RBAC: Authorizing `identity:get_project(project_id=08405881684e4a9fa222268105021f1a)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:16.199405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb38fa1-a0b3-490d-b4ae-e61dafb85801 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:16.201059 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 554/1096] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 06:01:16 2022] GET /identity/v3/projects/08405881684e4a9fa222268105021f1a => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:16.207116 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb38fa1-a0b3-490d-b4ae-e61dafb85801 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:16.224039 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7be02c-d1be-400f-b7ec-781639fb58ed None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:16.243780 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:16.258638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5d907-e45e-4402-bbf0-edd0aa659f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:16.259240 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5d907-e45e-4402-bbf0-edd0aa659f5f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:16.259513 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5d907-e45e-4402-bbf0-edd0aa659f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:16.259776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5d907-e45e-4402-bbf0-edd0aa659f5f None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:16.260402 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5d907-e45e-4402-bbf0-edd0aa659f5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:16.260692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5d907-e45e-4402-bbf0-edd0aa659f5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:16.262047 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5d907-e45e-4402-bbf0-edd0aa659f5f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:16.273499 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 543/1097] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:16.279823 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5d907-e45e-4402-bbf0-edd0aa659f5f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:16.311705 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb38fa1-a0b3-490d-b4ae-e61dafb85801 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:16.345769 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae222b1-5301-4cda-a71b-259ac4df3cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:16.346363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae222b1-5301-4cda-a71b-259ac4df3cfa None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:16.346675 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae222b1-5301-4cda-a71b-259ac4df3cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:16.346942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae222b1-5301-4cda-a71b-259ac4df3cfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:16.347649 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae222b1-5301-4cda-a71b-259ac4df3cfa None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=08405881684e4a9fa222268105021f1a, user.id=20b1751bc12b4dca85ef790bcb217c2e) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:16.347949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae222b1-5301-4cda-a71b-259ac4df3cfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:16.349358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae222b1-5301-4cda-a71b-259ac4df3cfa None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:16.355220 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 555/1098] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:01:16 2022] GET /identity/v3/role_assignments?user.id=20b1751bc12b4dca85ef790bcb217c2e&scope.project.id=08405881684e4a9fa222268105021f1a&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:16.376403 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:16.383578 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5d907-e45e-4402-bbf0-edd0aa659f5f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:16.423718 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000e9f2-154a-42c5-b2b6-69687d6d8043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:16.424369 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000e9f2-154a-42c5-b2b6-69687d6d8043 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:16.424648 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000e9f2-154a-42c5-b2b6-69687d6d8043 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:16.424919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000e9f2-154a-42c5-b2b6-69687d6d8043 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:16.433436 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000e9f2-154a-42c5-b2b6-69687d6d8043 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:16.435161 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000e9f2-154a-42c5-b2b6-69687d6d8043 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-8000e9f2-154a-42c5-b2b6-69687d6d8043 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:01:16.447691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:16.449443 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:16.450428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:16.450428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:16.450428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:01:16.450428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:01:16.450428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:01:16.450428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:01:16.450428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:01:16.450428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:01:16.450428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 06:01:16.450428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:16.450428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 544/1099] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Aug 23 06:01:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:01:16.455771 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae222b1-5301-4cda-a71b-259ac4df3cfa None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:16.557607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:16.561634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae222b1-5301-4cda-a71b-259ac4df3cfa None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:16.595577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968095d8-cc93-426d-bc97-6f38aa7453e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:16.596111 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968095d8-cc93-426d-bc97-6f38aa7453e8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:16.596391 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968095d8-cc93-426d-bc97-6f38aa7453e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:16.596663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968095d8-cc93-426d-bc97-6f38aa7453e8 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:16.597396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968095d8-cc93-426d-bc97-6f38aa7453e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:16.597700 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968095d8-cc93-426d-bc97-6f38aa7453e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:16.599181 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968095d8-cc93-426d-bc97-6f38aa7453e8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:16.616859 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 556/1100] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Aug 23 06:01:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:16.623791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000e9f2-154a-42c5-b2b6-69687d6d8043 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:16.723167 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000e9f2-154a-42c5-b2b6-69687d6d8043 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:16.758137 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c58499-90f0-4af1-94e8-ac8eaef71ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:16.758765 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c58499-90f0-4af1-94e8-ac8eaef71ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:16.759042 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c58499-90f0-4af1-94e8-ac8eaef71ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:16.759307 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c58499-90f0-4af1-94e8-ac8eaef71ca8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:16.783417 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:16.827245 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c58499-90f0-4af1-94e8-ac8eaef71ca8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:16.828692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c58499-90f0-4af1-94e8-ac8eaef71ca8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:16.928094 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 545/1101] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:01:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:01:16.934429 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968095d8-cc93-426d-bc97-6f38aa7453e8 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:17.042158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968095d8-cc93-426d-bc97-6f38aa7453e8 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:17.076167 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76890a-b342-4856-afe9-c35ff8aa1c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:17.076736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76890a-b342-4856-afe9-c35ff8aa1c0b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:17.077010 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76890a-b342-4856-afe9-c35ff8aa1c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:17.077276 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76890a-b342-4856-afe9-c35ff8aa1c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:17.087450 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76890a-b342-4856-afe9-c35ff8aa1c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:17.088887 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76890a-b342-4856-afe9-c35ff8aa1c0b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-fd76890a-b342-4856-afe9-c35ff8aa1c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:01:17.104553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:17.106246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:01:17.107733 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:01:17.107733 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:01:17.107733 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:17.107733 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 557/1102] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:01:17.111898 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c58499-90f0-4af1-94e8-ac8eaef71ca8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:17.166476 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:17.209957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c58499-90f0-4af1-94e8-ac8eaef71ca8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:17.243776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7da645-e78f-4c5c-8962-dfcac6aa205b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:17.244376 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7da645-e78f-4c5c-8962-dfcac6aa205b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:17.244784 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7da645-e78f-4c5c-8962-dfcac6aa205b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:17.244998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7da645-e78f-4c5c-8962-dfcac6aa205b None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:17.246257 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7da645-e78f-4c5c-8962-dfcac6aa205b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:17.246727 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7da645-e78f-4c5c-8962-dfcac6aa205b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:17.248188 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7da645-e78f-4c5c-8962-dfcac6aa205b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:17.260003 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 546/1103] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:17.266344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76890a-b342-4856-afe9-c35ff8aa1c0b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:17.279925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:17.368918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76890a-b342-4856-afe9-c35ff8aa1c0b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:17.403450 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194a2ad-4f2f-4ef3-806e-899b5e95e283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:17.404081 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194a2ad-4f2f-4ef3-806e-899b5e95e283 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:17.404361 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194a2ad-4f2f-4ef3-806e-899b5e95e283 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:17.404632 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194a2ad-4f2f-4ef3-806e-899b5e95e283 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:17.411268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194a2ad-4f2f-4ef3-806e-899b5e95e283 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:17.412707 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194a2ad-4f2f-4ef3-806e-899b5e95e283 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:17.424258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-7194a2ad-4f2f-4ef3-806e-899b5e95e283 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:01:17.424258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:17.424258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:17.424258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:17.424258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:17.424258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:17.424258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:17.424258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:17.424258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:17.424258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:17.424258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:17.424258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:17.424258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:01:17.424258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:01:17.424258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:17.425319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:17.426478 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:01:17.426478 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:01:17.426478 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:01:17.426478 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:01:17.426478 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:01:17.426478 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:01:17.426478 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:01:17.426478 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:17.430190 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 558/1104] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:01:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:17.431225 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7da645-e78f-4c5c-8962-dfcac6aa205b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:17.533757 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7da645-e78f-4c5c-8962-dfcac6aa205b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:17.550499 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:17.568793 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797cfea-071e-47da-927d-071d2b7698a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:17.569393 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797cfea-071e-47da-927d-071d2b7698a1 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:17.569669 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797cfea-071e-47da-927d-071d2b7698a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:17.569940 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797cfea-071e-47da-927d-071d2b7698a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:17.570798 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797cfea-071e-47da-927d-071d2b7698a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:17.570954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797cfea-071e-47da-927d-071d2b7698a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:17.572382 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797cfea-071e-47da-927d-071d2b7698a1 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:17.584596 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 547/1105] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:01:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:17.590771 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194a2ad-4f2f-4ef3-806e-899b5e95e283 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:17.677134 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:17.716783 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194a2ad-4f2f-4ef3-806e-899b5e95e283 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:17.752480 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b80ddd-9681-41bb-82af-65dc9143cf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:17.753083 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b80ddd-9681-41bb-82af-65dc9143cf26 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:17.753360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b80ddd-9681-41bb-82af-65dc9143cf26 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:17.753629 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b80ddd-9681-41bb-82af-65dc9143cf26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:17.754380 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b80ddd-9681-41bb-82af-65dc9143cf26 None admin] RBAC: Adding query filter params (role.id=aa12d72368054e13a59fa2a814d51afd, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=91267a049343452eb208205ee3ea27c4) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:17.754717 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b80ddd-9681-41bb-82af-65dc9143cf26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:17.756173 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b80ddd-9681-41bb-82af-65dc9143cf26 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:17.766591 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 559/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:01:17 2022] GET /identity/v3/role_assignments?user.id=91267a049343452eb208205ee3ea27c4&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=aa12d72368054e13a59fa2a814d51afd => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:18.313847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991af31-de7d-4f0a-a989-8c33eef9b76b None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:18.314397 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991af31-de7d-4f0a-a989-8c33eef9b76b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:18.314397 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991af31-de7d-4f0a-a989-8c33eef9b76b None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:18.315948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 548/1107] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:18.322027 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812b1c7-5ecd-4a70-9e01-aad1284be44b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:18.322505 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812b1c7-5ecd-4a70-9e01-aad1284be44b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:18.322637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812b1c7-5ecd-4a70-9e01-aad1284be44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:18.344582 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-a812b1c7-5ecd-4a70-9e01-aad1284be44b None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:18.345798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a812b1c7-5ecd-4a70-9e01-aad1284be44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:18.435685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 560/1108] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:18.709894 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991af31-de7d-4f0a-a989-8c33eef9b76b None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:18.820609 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991af31-de7d-4f0a-a989-8c33eef9b76b None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:18.858179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd961e9-526d-4b04-a790-d4cc78bbd596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:18.858835 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd961e9-526d-4b04-a790-d4cc78bbd596 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:18.859112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd961e9-526d-4b04-a790-d4cc78bbd596 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:18.859379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd961e9-526d-4b04-a790-d4cc78bbd596 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:18.860503 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd961e9-526d-4b04-a790-d4cc78bbd596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:18.862304 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd961e9-526d-4b04-a790-d4cc78bbd596 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:18.863963 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 549/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:18.869984 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812b1c7-5ecd-4a70-9e01-aad1284be44b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:18.933575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:18.977960 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812b1c7-5ecd-4a70-9e01-aad1284be44b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:19.012053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0179ab8-fc8b-42eb-9507-3820ef5348df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:19.012649 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0179ab8-fc8b-42eb-9507-3820ef5348df None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:19.012926 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0179ab8-fc8b-42eb-9507-3820ef5348df None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:19.013192 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0179ab8-fc8b-42eb-9507-3820ef5348df None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:19.013831 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0179ab8-fc8b-42eb-9507-3820ef5348df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:19.015282 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0179ab8-fc8b-42eb-9507-3820ef5348df None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:19.037789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 561/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 06:01:18 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:01:21.086143 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42061352-0eeb-47e1-8d82-dcb0867e4a98 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:21.086722 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42061352-0eeb-47e1-8d82-dcb0867e4a98 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:21.086722 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42061352-0eeb-47e1-8d82-dcb0867e4a98 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:21.088072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 550/1111] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:21.093779 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f592f-a7ca-41e6-9caf-7e0d2560be4b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:21.094228 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f592f-a7ca-41e6-9caf-7e0d2560be4b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:21.094335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f592f-a7ca-41e6-9caf-7e0d2560be4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:21.113988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-416f592f-a7ca-41e6-9caf-7e0d2560be4b None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:21.115514 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-416f592f-a7ca-41e6-9caf-7e0d2560be4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:21.187949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 562/1112] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:21.473759 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42061352-0eeb-47e1-8d82-dcb0867e4a98 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:21.522966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:21.581100 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42061352-0eeb-47e1-8d82-dcb0867e4a98 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:21.614710 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7036a5-124a-4dca-9a05-ec98dca32f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:21.615336 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7036a5-124a-4dca-9a05-ec98dca32f9f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:21.615609 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7036a5-124a-4dca-9a05-ec98dca32f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:21.615874 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7036a5-124a-4dca-9a05-ec98dca32f9f None admin] PATH_INFO: `/v3/roles/f47755365f24477f9846225c9e91de2d` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:21.617264 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7036a5-124a-4dca-9a05-ec98dca32f9f None admin] RBAC: Authorizing `identity:get_role(role_id=f47755365f24477f9846225c9e91de2d)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:21.618924 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7036a5-124a-4dca-9a05-ec98dca32f9f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:21.620798 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 551/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 06:01:21 2022] GET /identity/v3/roles/f47755365f24477f9846225c9e91de2d => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:21.626964 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f592f-a7ca-41e6-9caf-7e0d2560be4b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:21.676176 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:21.726494 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f592f-a7ca-41e6-9caf-7e0d2560be4b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:21.762442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4818f8-ccd4-4c30-9cde-146c6ccf34c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:21.763131 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4818f8-ccd4-4c30-9cde-146c6ccf34c4 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:21.763408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4818f8-ccd4-4c30-9cde-146c6ccf34c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:21.763676 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4818f8-ccd4-4c30-9cde-146c6ccf34c4 None admin] PATH_INFO: `/v3/users/a29ddb7ab0654243a807314a5d1a2609` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:21.780324 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4818f8-ccd4-4c30-9cde-146c6ccf34c4 None admin] RBAC: Authorizing `identity:get_user(user_id=a29ddb7ab0654243a807314a5d1a2609)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:21.781792 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4818f8-ccd4-4c30-9cde-146c6ccf34c4 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:21.783502 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 563/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 06:01:21 2022] GET /identity/v3/users/a29ddb7ab0654243a807314a5d1a2609 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:21.789796 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7036a5-124a-4dca-9a05-ec98dca32f9f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:21.893082 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7036a5-124a-4dca-9a05-ec98dca32f9f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:21.907995 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:21.930602 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352238e-4607-4e88-9cde-0eb114a265ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:21.931212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352238e-4607-4e88-9cde-0eb114a265ad None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:21.931485 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352238e-4607-4e88-9cde-0eb114a265ad None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:21.931750 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352238e-4607-4e88-9cde-0eb114a265ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:22.009052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7352238e-4607-4e88-9cde-0eb114a265ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:22.010525 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7352238e-4607-4e88-9cde-0eb114a265ad None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:22.109197 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 552/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:01:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:01:22.116232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4818f8-ccd4-4c30-9cde-146c6ccf34c4 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:22.222414 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4818f8-ccd4-4c30-9cde-146c6ccf34c4 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:22.232428 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:22.259864 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed3774-8165-4f20-a199-d51928a93720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:22.260417 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed3774-8165-4f20-a199-d51928a93720 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:22.260695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed3774-8165-4f20-a199-d51928a93720 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:22.261668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed3774-8165-4f20-a199-d51928a93720 None admin] PATH_INFO: `/v3/projects/08405881684e4a9fa222268105021f1a` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:22.262932 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed3774-8165-4f20-a199-d51928a93720 None admin] RBAC: Authorizing `identity:get_project(project_id=08405881684e4a9fa222268105021f1a)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:22.264368 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed3774-8165-4f20-a199-d51928a93720 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:22.266008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 564/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 06:01:22 2022] GET /identity/v3/projects/08405881684e4a9fa222268105021f1a => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:22.272662 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352238e-4607-4e88-9cde-0eb114a265ad None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:22.287680 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:22.374041 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352238e-4607-4e88-9cde-0eb114a265ad None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:22.408255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb4ea9-6be9-44df-bc28-60b7fbce62c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:22.408849 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb4ea9-6be9-44df-bc28-60b7fbce62c5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:22.409125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb4ea9-6be9-44df-bc28-60b7fbce62c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:22.409393 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb4ea9-6be9-44df-bc28-60b7fbce62c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:22.410109 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb4ea9-6be9-44df-bc28-60b7fbce62c5 None admin] RBAC: Adding query filter params (role.id=f47755365f24477f9846225c9e91de2d, scope.project.id=08405881684e4a9fa222268105021f1a, user.id=a29ddb7ab0654243a807314a5d1a2609) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:22.410411 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb4ea9-6be9-44df-bc28-60b7fbce62c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:22.411857 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb4ea9-6be9-44df-bc28-60b7fbce62c5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:22.417255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 553/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:01:22 2022] GET /identity/v3/role_assignments?user.id=a29ddb7ab0654243a807314a5d1a2609&scope.project.id=08405881684e4a9fa222268105021f1a&role.id=f47755365f24477f9846225c9e91de2d => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:24.356852 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fc60c-003c-4d26-b802-10b3fe3e239e None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:24.357417 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fc60c-003c-4d26-b802-10b3fe3e239e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:24.357417 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fc60c-003c-4d26-b802-10b3fe3e239e None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:24.358784 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 565/1118] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:24.364080 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf2320-c9ad-41cb-84dd-f5a3a14a039a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:24.364510 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf2320-c9ad-41cb-84dd-f5a3a14a039a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:24.364747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf2320-c9ad-41cb-84dd-f5a3a14a039a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:24.383666 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-88bf2320-c9ad-41cb-84dd-f5a3a14a039a None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:24.384863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-88bf2320-c9ad-41cb-84dd-f5a3a14a039a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:24.450923 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 554/1119] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:24.749714 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fc60c-003c-4d26-b802-10b3fe3e239e None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:24.861362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fc60c-003c-4d26-b802-10b3fe3e239e None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:24.906391 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760ad92-9319-481e-b089-593298ad94a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:24.907375 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760ad92-9319-481e-b089-593298ad94a2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:24.907815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760ad92-9319-481e-b089-593298ad94a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:24.908125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760ad92-9319-481e-b089-593298ad94a2 None admin] PATH_INFO: `/v3/roles/f47755365f24477f9846225c9e91de2d` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:24.910198 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760ad92-9319-481e-b089-593298ad94a2 None admin] RBAC: Authorizing `identity:get_role(role_id=f47755365f24477f9846225c9e91de2d)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:24.912393 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760ad92-9319-481e-b089-593298ad94a2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:24.914551 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 566/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 06:01:24 2022] GET /identity/v3/roles/f47755365f24477f9846225c9e91de2d => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:24.926621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf2320-c9ad-41cb-84dd-f5a3a14a039a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:24.935590 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:25.031492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf2320-c9ad-41cb-84dd-f5a3a14a039a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:25.065993 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb25a9b-10ef-43c6-93a1-440d30625e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:25.066638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb25a9b-10ef-43c6-93a1-440d30625e25 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:25.066921 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb25a9b-10ef-43c6-93a1-440d30625e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:25.067183 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb25a9b-10ef-43c6-93a1-440d30625e25 None admin] PATH_INFO: `/v3/users/a29ddb7ab0654243a807314a5d1a2609` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:25.068305 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb25a9b-10ef-43c6-93a1-440d30625e25 None admin] RBAC: Authorizing `identity:get_user(user_id=a29ddb7ab0654243a807314a5d1a2609)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:25.069731 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb25a9b-10ef-43c6-93a1-440d30625e25 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:25.071335 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 555/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 06:01:24 2022] GET /identity/v3/users/a29ddb7ab0654243a807314a5d1a2609 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:25.077204 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760ad92-9319-481e-b089-593298ad94a2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:25.122851 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:25.176917 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760ad92-9319-481e-b089-593298ad94a2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:25.217421 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123db085-3c59-42d9-80a1-e7b817fb348d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:25.218498 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123db085-3c59-42d9-80a1-e7b817fb348d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:25.218919 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123db085-3c59-42d9-80a1-e7b817fb348d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:25.219197 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123db085-3c59-42d9-80a1-e7b817fb348d None admin] PATH_INFO: `/v3/projects/08405881684e4a9fa222268105021f1a` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:25.220393 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123db085-3c59-42d9-80a1-e7b817fb348d None admin] RBAC: Authorizing `identity:get_project(project_id=08405881684e4a9fa222268105021f1a)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:25.221850 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123db085-3c59-42d9-80a1-e7b817fb348d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:25.228522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 567/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 06:01:25 2022] GET /identity/v3/projects/08405881684e4a9fa222268105021f1a => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:25.230963 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb25a9b-10ef-43c6-93a1-440d30625e25 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:25.315897 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:25.342215 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb25a9b-10ef-43c6-93a1-440d30625e25 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:25.378388 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41b1f5-d09e-4755-9cab-6f8ce4fdf774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:25.379117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41b1f5-d09e-4755-9cab-6f8ce4fdf774 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:25.379340 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41b1f5-d09e-4755-9cab-6f8ce4fdf774 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:25.379610 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41b1f5-d09e-4755-9cab-6f8ce4fdf774 None admin] PATH_INFO: `/v3/projects/08405881684e4a9fa222268105021f1a/users/a29ddb7ab0654243a807314a5d1a2609/roles/f47755365f24477f9846225c9e91de2d` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:25.381760 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41b1f5-d09e-4755-9cab-6f8ce4fdf774 None admin] RBAC: Authorizing `identity:create_grant(project_id=08405881684e4a9fa222268105021f1a, user_id=a29ddb7ab0654243a807314a5d1a2609, role_id=f47755365f24477f9846225c9e91de2d)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:25.383228 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41b1f5-d09e-4755-9cab-6f8ce4fdf774 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:25.396827 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 556/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:01:25 2022] PUT /identity/v3/projects/08405881684e4a9fa222268105021f1a/users/a29ddb7ab0654243a807314a5d1a2609/roles/f47755365f24477f9846225c9e91de2d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 06:01:27.310960 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b7299-a954-4935-9fc7-a0843ba145de None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:27.311490 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b7299-a954-4935-9fc7-a0843ba145de None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:27.311490 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b7299-a954-4935-9fc7-a0843ba145de None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:27.312829 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 568/1124] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:27.318014 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c3368-cf21-4eee-9444-b403499f32b3 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:27.318418 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c3368-cf21-4eee-9444-b403499f32b3 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:27.318576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c3368-cf21-4eee-9444-b403499f32b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:27.336204 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-ae6c3368-cf21-4eee-9444-b403499f32b3 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:27.337378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ae6c3368-cf21-4eee-9444-b403499f32b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:27.413175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 557/1125] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:27.680601 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b7299-a954-4935-9fc7-a0843ba145de None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:27.785177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b7299-a954-4935-9fc7-a0843ba145de None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:27.799400 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:27.820513 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65198873-328a-4c81-b359-b602a640f09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:27.821132 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65198873-328a-4c81-b359-b602a640f09d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:27.821410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65198873-328a-4c81-b359-b602a640f09d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:27.821681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65198873-328a-4c81-b359-b602a640f09d None admin] PATH_INFO: `/v3/roles/f47755365f24477f9846225c9e91de2d` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:27.823083 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65198873-328a-4c81-b359-b602a640f09d None admin] RBAC: Authorizing `identity:get_role(role_id=f47755365f24477f9846225c9e91de2d)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:27.824977 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65198873-328a-4c81-b359-b602a640f09d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:27.826301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 569/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 06:01:27 2022] GET /identity/v3/roles/f47755365f24477f9846225c9e91de2d => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:27.832212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c3368-cf21-4eee-9444-b403499f32b3 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:27.893952 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:27.934657 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c3368-cf21-4eee-9444-b403499f32b3 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:27.971018 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087525ef-3bff-46d3-9890-d854ed185fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:27.971652 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087525ef-3bff-46d3-9890-d854ed185fdc None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:27.971931 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087525ef-3bff-46d3-9890-d854ed185fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:27.972200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087525ef-3bff-46d3-9890-d854ed185fdc None admin] PATH_INFO: `/v3/users/a29ddb7ab0654243a807314a5d1a2609` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:27.973314 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087525ef-3bff-46d3-9890-d854ed185fdc None admin] RBAC: Authorizing `identity:get_user(user_id=a29ddb7ab0654243a807314a5d1a2609)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:27.974768 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087525ef-3bff-46d3-9890-d854ed185fdc None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:27.976361 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 558/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 06:01:27 2022] GET /identity/v3/users/a29ddb7ab0654243a807314a5d1a2609 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:27.982924 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65198873-328a-4c81-b359-b602a640f09d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:28.087004 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65198873-328a-4c81-b359-b602a640f09d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:28.122199 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59284ffc-6592-4db4-b220-5deca3c3a6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:28.122841 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59284ffc-6592-4db4-b220-5deca3c3a6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:28.123116 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59284ffc-6592-4db4-b220-5deca3c3a6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:28.123384 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59284ffc-6592-4db4-b220-5deca3c3a6d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:28.171700 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:28.191792 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59284ffc-6592-4db4-b220-5deca3c3a6d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:28.193153 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59284ffc-6592-4db4-b220-5deca3c3a6d2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:28.296985 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 570/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:01:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 06:01:28.302864 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087525ef-3bff-46d3-9890-d854ed185fdc None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:28.402685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087525ef-3bff-46d3-9890-d854ed185fdc None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:28.430694 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:28.436958 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce04d7-21f6-4765-8bb4-9465feaa25ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:28.437585 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce04d7-21f6-4765-8bb4-9465feaa25ee None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:28.437860 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce04d7-21f6-4765-8bb4-9465feaa25ee None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:28.438135 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce04d7-21f6-4765-8bb4-9465feaa25ee None admin] PATH_INFO: `/v3/projects/08405881684e4a9fa222268105021f1a` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:28.439268 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce04d7-21f6-4765-8bb4-9465feaa25ee None admin] RBAC: Authorizing `identity:get_project(project_id=08405881684e4a9fa222268105021f1a)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:28.440691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce04d7-21f6-4765-8bb4-9465feaa25ee None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:28.442387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 559/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 06:01:28 2022] GET /identity/v3/projects/08405881684e4a9fa222268105021f1a => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:28.449072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59284ffc-6592-4db4-b220-5deca3c3a6d2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:28.529840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:28.549784 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59284ffc-6592-4db4-b220-5deca3c3a6d2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:28.593457 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570cb2f-5673-4808-82c7-705322afda5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:28.594055 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570cb2f-5673-4808-82c7-705322afda5b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:28.594333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570cb2f-5673-4808-82c7-705322afda5b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:28.594636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570cb2f-5673-4808-82c7-705322afda5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:28.595356 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570cb2f-5673-4808-82c7-705322afda5b None admin] RBAC: Adding query filter params (role.id=f47755365f24477f9846225c9e91de2d, scope.project.id=08405881684e4a9fa222268105021f1a, user.id=a29ddb7ab0654243a807314a5d1a2609) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:28.595660 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570cb2f-5673-4808-82c7-705322afda5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:28.597067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570cb2f-5673-4808-82c7-705322afda5b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:28.602699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 571/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:01:28 2022] GET /identity/v3/role_assignments?user.id=a29ddb7ab0654243a807314a5d1a2609&scope.project.id=08405881684e4a9fa222268105021f1a&role.id=f47755365f24477f9846225c9e91de2d => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:30.558699 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60e2d9-a6ff-4138-8b40-40197b6d505f None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:30.559577 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60e2d9-a6ff-4138-8b40-40197b6d505f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:30.559864 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60e2d9-a6ff-4138-8b40-40197b6d505f None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:30.561262 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 560/1131] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:30.566760 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90c067-4d23-423e-9d26-bc44a4816c4f None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:30.567166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90c067-4d23-423e-9d26-bc44a4816c4f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:30.567298 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90c067-4d23-423e-9d26-bc44a4816c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:30.585461 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1d90c067-4d23-423e-9d26-bc44a4816c4f None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:30.586686 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1d90c067-4d23-423e-9d26-bc44a4816c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:30.653140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 572/1132] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:30.928323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60e2d9-a6ff-4138-8b40-40197b6d505f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:31.043399 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60e2d9-a6ff-4138-8b40-40197b6d505f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:31.090501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000b6ac-0ab0-4778-a69b-6089470399f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:31.091489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000b6ac-0ab0-4778-a69b-6089470399f6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:31.092229 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000b6ac-0ab0-4778-a69b-6089470399f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:31.092229 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000b6ac-0ab0-4778-a69b-6089470399f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:31.093695 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000b6ac-0ab0-4778-a69b-6089470399f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:31.096076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000b6ac-0ab0-4778-a69b-6089470399f6 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:31.098430 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 561/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:31.106186 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90c067-4d23-423e-9d26-bc44a4816c4f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:31.180590 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:31.205765 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90c067-4d23-423e-9d26-bc44a4816c4f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:31.239916 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8361c-0140-4085-8924-240688ced26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:31.240520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8361c-0140-4085-8924-240688ced26d None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:31.240798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8361c-0140-4085-8924-240688ced26d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:31.241066 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8361c-0140-4085-8924-240688ced26d None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:31.241700 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8361c-0140-4085-8924-240688ced26d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:31.243158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8361c-0140-4085-8924-240688ced26d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:31.271792 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 573/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 23 06:01:31 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:01:33.234005 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924e5bf-fcb4-4144-9203-f78a4f180e79 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:33.234528 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924e5bf-fcb4-4144-9203-f78a4f180e79 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:33.234528 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924e5bf-fcb4-4144-9203-f78a4f180e79 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:33.235828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 562/1135] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:33.241483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befc3e7-a961-40ed-a97e-e73556f823c0 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:33.241924 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befc3e7-a961-40ed-a97e-e73556f823c0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:33.242102 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befc3e7-a961-40ed-a97e-e73556f823c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:33.262153 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-3befc3e7-a961-40ed-a97e-e73556f823c0 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:33.263396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3befc3e7-a961-40ed-a97e-e73556f823c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:33.330881 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 574/1136] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:01:33.605188 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924e5bf-fcb4-4144-9203-f78a4f180e79 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:33.674140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:33.708869 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924e5bf-fcb4-4144-9203-f78a4f180e79 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:33.743846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14c813-292d-495b-9387-975c078fff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:33.744488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14c813-292d-495b-9387-975c078fff3d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:33.744760 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14c813-292d-495b-9387-975c078fff3d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:33.745097 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14c813-292d-495b-9387-975c078fff3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:33.746361 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14c813-292d-495b-9387-975c078fff3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:33.747841 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14c813-292d-495b-9387-975c078fff3d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:33.749475 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 563/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:01:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:33.755644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befc3e7-a961-40ed-a97e-e73556f823c0 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:33.811597 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:33.855734 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befc3e7-a961-40ed-a97e-e73556f823c0 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:33.890072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05655beb-4d57-461a-878d-edf92aecf6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:33.890702 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05655beb-4d57-461a-878d-edf92aecf6f5 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:33.890988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05655beb-4d57-461a-878d-edf92aecf6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:33.891256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05655beb-4d57-461a-878d-edf92aecf6f5 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:33.891904 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05655beb-4d57-461a-878d-edf92aecf6f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:33.893329 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05655beb-4d57-461a-878d-edf92aecf6f5 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:33.923733 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 575/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 06:01:33 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:01:35.876672 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43f300-835e-4c6c-ad69-3897922f991f None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:35.877221 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43f300-835e-4c6c-ad69-3897922f991f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:35.877221 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43f300-835e-4c6c-ad69-3897922f991f None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:35.878565 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 564/1139] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:35.884229 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8764a9-542b-4e4e-8039-3d7fc20f670f None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:35.884771 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8764a9-542b-4e4e-8039-3d7fc20f670f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:35.884868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8764a9-542b-4e4e-8039-3d7fc20f670f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:35.904673 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-5f8764a9-542b-4e4e-8039-3d7fc20f670f None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:35.905894 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5f8764a9-542b-4e4e-8039-3d7fc20f670f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:35.977285 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 576/1140] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:36.250673 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43f300-835e-4c6c-ad69-3897922f991f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:36.351845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43f300-835e-4c6c-ad69-3897922f991f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:36.388967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f454031f-16be-4a84-9c70-9dfabd791217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:36.389593 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454031f-16be-4a84-9c70-9dfabd791217 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:36.389874 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454031f-16be-4a84-9c70-9dfabd791217 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:36.390211 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454031f-16be-4a84-9c70-9dfabd791217 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:36.393019 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:36.401949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f454031f-16be-4a84-9c70-9dfabd791217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-f454031f-16be-4a84-9c70-9dfabd791217 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:01:36.406966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:36.407941 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:36.408983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:36.408983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:36.408983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:36.408983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:36.408983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:01:36.408983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:01:36.408983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:01:36.408983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:01:36.408983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:01:36.408983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:01:36.408983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:01:36.408983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:36.408983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 565/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:01:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:36.414170 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8764a9-542b-4e4e-8039-3d7fc20f670f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:36.456934 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:36.521204 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8764a9-542b-4e4e-8039-3d7fc20f670f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:36.555836 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78f96b-7fe4-461d-bf97-7cd602ad7bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:36.556451 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78f96b-7fe4-461d-bf97-7cd602ad7bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:36.556732 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78f96b-7fe4-461d-bf97-7cd602ad7bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:36.557005 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78f96b-7fe4-461d-bf97-7cd602ad7bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:36.557720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78f96b-7fe4-461d-bf97-7cd602ad7bd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:36.558083 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78f96b-7fe4-461d-bf97-7cd602ad7bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:36.559422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78f96b-7fe4-461d-bf97-7cd602ad7bd3 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:36.567635 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 577/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:01:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:36.573543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f454031f-16be-4a84-9c70-9dfabd791217 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:36.676557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f454031f-16be-4a84-9c70-9dfabd791217 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:36.714284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d3ba2-9fa4-4e9c-b595-d80c191788a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:36.714944 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d3ba2-9fa4-4e9c-b595-d80c191788a8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:36.715223 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d3ba2-9fa4-4e9c-b595-d80c191788a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:36.715492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d3ba2-9fa4-4e9c-b595-d80c191788a8 None admin] PATH_INFO: `/v3/users/5cc914b4df9b47eab2298e3538d3c843` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:36.729660 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d3ba2-9fa4-4e9c-b595-d80c191788a8 None admin] RBAC: Authorizing `identity:get_user(user_id=5cc914b4df9b47eab2298e3538d3c843)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:36.731150 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d3ba2-9fa4-4e9c-b595-d80c191788a8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:36.732811 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 566/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 06:01:36 2022] GET /identity/v3/users/5cc914b4df9b47eab2298e3538d3c843 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:36.738803 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78f96b-7fe4-461d-bf97-7cd602ad7bd3 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:36.840461 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78f96b-7fe4-461d-bf97-7cd602ad7bd3 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:36.842953 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:36.877399 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931be60-f1ea-4994-8be9-6edda03f904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:36.878004 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931be60-f1ea-4994-8be9-6edda03f904a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:36.878285 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931be60-f1ea-4994-8be9-6edda03f904a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:36.878576 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931be60-f1ea-4994-8be9-6edda03f904a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:36.947534 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931be60-f1ea-4994-8be9-6edda03f904a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:36.949501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931be60-f1ea-4994-8be9-6edda03f904a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:37.055529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 578/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:01:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:01:37.056718 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d3ba2-9fa4-4e9c-b595-d80c191788a8 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:37.114160 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:37.162460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d3ba2-9fa4-4e9c-b595-d80c191788a8 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:37.197473 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5772e-0940-45ba-b755-cdf381a7be30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:37.199311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5772e-0940-45ba-b755-cdf381a7be30 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:37.199599 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5772e-0940-45ba-b755-cdf381a7be30 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:37.199874 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5772e-0940-45ba-b755-cdf381a7be30 None admin] PATH_INFO: `/v3/projects/09389d8f963143508d25e3773cf92e61` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:37.201004 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5772e-0940-45ba-b755-cdf381a7be30 None admin] RBAC: Authorizing `identity:get_project(project_id=09389d8f963143508d25e3773cf92e61)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:37.202445 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5772e-0940-45ba-b755-cdf381a7be30 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:37.204131 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 567/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 06:01:37 2022] GET /identity/v3/projects/09389d8f963143508d25e3773cf92e61 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:37.210273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931be60-f1ea-4994-8be9-6edda03f904a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:37.214222 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:37.311083 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931be60-f1ea-4994-8be9-6edda03f904a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:37.345994 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c12f76-c795-49b0-854a-199596444036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:37.346630 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c12f76-c795-49b0-854a-199596444036 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:37.346914 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c12f76-c795-49b0-854a-199596444036 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:37.347186 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c12f76-c795-49b0-854a-199596444036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:37.347957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c12f76-c795-49b0-854a-199596444036 None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=09389d8f963143508d25e3773cf92e61, user.id=5cc914b4df9b47eab2298e3538d3c843) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:37.348222 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c12f76-c795-49b0-854a-199596444036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:37.349658 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c12f76-c795-49b0-854a-199596444036 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:37.356173 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 579/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:01:37 2022] GET /identity/v3/role_assignments?user.id=5cc914b4df9b47eab2298e3538d3c843&scope.project.id=09389d8f963143508d25e3773cf92e61&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:39.317758 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802a304-ce05-49a0-82ef-3a6fe88968a2 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:39.318351 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802a304-ce05-49a0-82ef-3a6fe88968a2 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:39.318351 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802a304-ce05-49a0-82ef-3a6fe88968a2 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:39.319677 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 568/1147] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:39.325401 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18215d-fdf0-4a2c-b541-bb6f4ed52f03 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:39.325820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18215d-fdf0-4a2c-b541-bb6f4ed52f03 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:39.325953 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18215d-fdf0-4a2c-b541-bb6f4ed52f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:39.345398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4b18215d-fdf0-4a2c-b541-bb6f4ed52f03 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:39.346621 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4b18215d-fdf0-4a2c-b541-bb6f4ed52f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:39.410615 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:39.416716 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 580/1148] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:39.687742 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802a304-ce05-49a0-82ef-3a6fe88968a2 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:39.790063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802a304-ce05-49a0-82ef-3a6fe88968a2 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:39.812451 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:39.824956 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b00cb-10a9-4f92-874c-3324d16b2e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:39.825610 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b00cb-10a9-4f92-874c-3324d16b2e19 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:39.825887 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b00cb-10a9-4f92-874c-3324d16b2e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:39.826154 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b00cb-10a9-4f92-874c-3324d16b2e19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:39.836247 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b00cb-10a9-4f92-874c-3324d16b2e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-d73b00cb-10a9-4f92-874c-3324d16b2e19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:01:39.841102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:39.842016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:39.842902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:39.842902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:39.842902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:39.842902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:39.842902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:01:39.842902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:01:39.842902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:01:39.842902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:01:39.842902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:01:39.842902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:01:39.842902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:01:39.842902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:39.842902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 569/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:01:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:01:39.849808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18215d-fdf0-4a2c-b541-bb6f4ed52f03 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:39.957524 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18215d-fdf0-4a2c-b541-bb6f4ed52f03 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:39.992027 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3513817-14f7-433c-8641-f383a2696972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:39.992630 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3513817-14f7-433c-8641-f383a2696972 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:39.992911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3513817-14f7-433c-8641-f383a2696972 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:39.993183 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3513817-14f7-433c-8641-f383a2696972 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:39.994105 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3513817-14f7-433c-8641-f383a2696972 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:39.994407 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3513817-14f7-433c-8641-f383a2696972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:39.996737 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3513817-14f7-433c-8641-f383a2696972 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:40.004773 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 581/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:01:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:01:40.010909 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b00cb-10a9-4f92-874c-3324d16b2e19 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:40.110693 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b00cb-10a9-4f92-874c-3324d16b2e19 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:40.145545 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd38d0-d10d-4a25-b30e-9808cb3e04e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:40.146164 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd38d0-d10d-4a25-b30e-9808cb3e04e2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:40.146439 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd38d0-d10d-4a25-b30e-9808cb3e04e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:40.146765 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd38d0-d10d-4a25-b30e-9808cb3e04e2 None admin] PATH_INFO: `/v3/users/5cc914b4df9b47eab2298e3538d3c843` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:40.147875 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd38d0-d10d-4a25-b30e-9808cb3e04e2 None admin] RBAC: Authorizing `identity:get_user(user_id=5cc914b4df9b47eab2298e3538d3c843)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:40.149293 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd38d0-d10d-4a25-b30e-9808cb3e04e2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:40.150906 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 570/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 06:01:40 2022] GET /identity/v3/users/5cc914b4df9b47eab2298e3538d3c843 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:40.156823 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3513817-14f7-433c-8641-f383a2696972 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:40.215239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:40.258362 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3513817-14f7-433c-8641-f383a2696972 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:40.292997 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e2372-c249-4efe-a670-4d0b96c5b38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:40.293632 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e2372-c249-4efe-a670-4d0b96c5b38c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:40.293911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e2372-c249-4efe-a670-4d0b96c5b38c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:40.294181 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e2372-c249-4efe-a670-4d0b96c5b38c None admin] PATH_INFO: `/v3/projects/09389d8f963143508d25e3773cf92e61` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:40.295378 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e2372-c249-4efe-a670-4d0b96c5b38c None admin] RBAC: Authorizing `identity:get_project(project_id=09389d8f963143508d25e3773cf92e61)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:40.296810 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e2372-c249-4efe-a670-4d0b96c5b38c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:40.298475 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 582/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 06:01:40 2022] GET /identity/v3/projects/09389d8f963143508d25e3773cf92e61 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:40.304473 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd38d0-d10d-4a25-b30e-9808cb3e04e2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:40.354305 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:40.405026 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd38d0-d10d-4a25-b30e-9808cb3e04e2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:40.441184 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af334acf-e552-4eb9-93db-5c516bc51a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:40.441783 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af334acf-e552-4eb9-93db-5c516bc51a43 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:40.442063 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af334acf-e552-4eb9-93db-5c516bc51a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:40.442334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af334acf-e552-4eb9-93db-5c516bc51a43 None admin] PATH_INFO: `/v3/projects/09389d8f963143508d25e3773cf92e61/users/5cc914b4df9b47eab2298e3538d3c843/roles/f6acb6fdca5a44de9ba00b2b0d4d3202` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:40.444271 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af334acf-e552-4eb9-93db-5c516bc51a43 None admin] RBAC: Authorizing `identity:create_grant(project_id=09389d8f963143508d25e3773cf92e61, user_id=5cc914b4df9b47eab2298e3538d3c843, role_id=f6acb6fdca5a44de9ba00b2b0d4d3202)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:40.445714 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af334acf-e552-4eb9-93db-5c516bc51a43 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:40.460119 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 571/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:01:40 2022] PUT /identity/v3/projects/09389d8f963143508d25e3773cf92e61/users/5cc914b4df9b47eab2298e3538d3c843/roles/f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 06:01:42.371678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c411e-eefb-40a4-8ed6-d67ef32aaed6 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:42.372209 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c411e-eefb-40a4-8ed6-d67ef32aaed6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:42.372209 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c411e-eefb-40a4-8ed6-d67ef32aaed6 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:42.373553 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 583/1154] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:42.378866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653adc29-9a9d-4478-96f0-ac7483ee9354 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:42.379295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653adc29-9a9d-4478-96f0-ac7483ee9354 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:42.379411 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653adc29-9a9d-4478-96f0-ac7483ee9354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:42.397733 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-653adc29-9a9d-4478-96f0-ac7483ee9354 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:42.398953 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-653adc29-9a9d-4478-96f0-ac7483ee9354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:42.465611 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 572/1155] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:42.739262 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c411e-eefb-40a4-8ed6-d67ef32aaed6 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:42.841126 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c411e-eefb-40a4-8ed6-d67ef32aaed6 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:42.867887 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:42.877490 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c0ddd-8043-40b3-93a1-549f1583e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:42.878120 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c0ddd-8043-40b3-93a1-549f1583e21d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:42.878395 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c0ddd-8043-40b3-93a1-549f1583e21d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:42.878699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c0ddd-8043-40b3-93a1-549f1583e21d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:42.890074 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c0ddd-8043-40b3-93a1-549f1583e21d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-160c0ddd-8043-40b3-93a1-549f1583e21d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:01:42.894808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:42.895722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:42.896564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:42.896564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:42.896564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:42.896564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:42.896564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:01:42.896564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:01:42.896564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:01:42.896564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:01:42.896564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:01:42.896564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:01:42.896564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:01:42.896564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:42.896564 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 584/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:01:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:42.902931 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653adc29-9a9d-4478-96f0-ac7483ee9354 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:42.948554 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:43.015169 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653adc29-9a9d-4478-96f0-ac7483ee9354 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:43.051475 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2f5c8-bc8c-499e-8b7c-ac59a97895d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:43.052076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2f5c8-bc8c-499e-8b7c-ac59a97895d2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:43.052354 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2f5c8-bc8c-499e-8b7c-ac59a97895d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:43.052628 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2f5c8-bc8c-499e-8b7c-ac59a97895d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:43.053310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2f5c8-bc8c-499e-8b7c-ac59a97895d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:43.053591 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2f5c8-bc8c-499e-8b7c-ac59a97895d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:43.055001 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2f5c8-bc8c-499e-8b7c-ac59a97895d2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:43.064817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 573/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:01:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:43.070759 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c0ddd-8043-40b3-93a1-549f1583e21d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:43.170737 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c0ddd-8043-40b3-93a1-549f1583e21d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:43.204276 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701744e6-aa95-4693-87b2-ed79e2f8b266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:43.204897 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701744e6-aa95-4693-87b2-ed79e2f8b266 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:43.205175 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701744e6-aa95-4693-87b2-ed79e2f8b266 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:43.205446 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701744e6-aa95-4693-87b2-ed79e2f8b266 None admin] PATH_INFO: `/v3/users/5cc914b4df9b47eab2298e3538d3c843` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:43.206580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701744e6-aa95-4693-87b2-ed79e2f8b266 None admin] RBAC: Authorizing `identity:get_user(user_id=5cc914b4df9b47eab2298e3538d3c843)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:43.208006 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701744e6-aa95-4693-87b2-ed79e2f8b266 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:43.209606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 585/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 06:01:43 2022] GET /identity/v3/users/5cc914b4df9b47eab2298e3538d3c843 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:43.216431 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2f5c8-bc8c-499e-8b7c-ac59a97895d2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:43.329866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2f5c8-bc8c-499e-8b7c-ac59a97895d2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:43.331411 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:43.365205 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50473ca-41d8-4172-94a2-0cb0dbb7db6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:43.365797 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50473ca-41d8-4172-94a2-0cb0dbb7db6e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:43.366072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50473ca-41d8-4172-94a2-0cb0dbb7db6e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:43.366340 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50473ca-41d8-4172-94a2-0cb0dbb7db6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:43.435575 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50473ca-41d8-4172-94a2-0cb0dbb7db6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:43.437047 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50473ca-41d8-4172-94a2-0cb0dbb7db6e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:43.540963 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 574/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:01:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:01:43.543094 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701744e6-aa95-4693-87b2-ed79e2f8b266 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:43.602001 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:43.649978 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701744e6-aa95-4693-87b2-ed79e2f8b266 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:43.684993 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4da820-3882-4a5c-97da-60a4e0ba208b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:43.685619 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4da820-3882-4a5c-97da-60a4e0ba208b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:43.685899 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4da820-3882-4a5c-97da-60a4e0ba208b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:43.686170 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4da820-3882-4a5c-97da-60a4e0ba208b None admin] PATH_INFO: `/v3/projects/09389d8f963143508d25e3773cf92e61` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:43.687331 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4da820-3882-4a5c-97da-60a4e0ba208b None admin] RBAC: Authorizing `identity:get_project(project_id=09389d8f963143508d25e3773cf92e61)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:43.688744 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4da820-3882-4a5c-97da-60a4e0ba208b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:43.690389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 586/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 06:01:43 2022] GET /identity/v3/projects/09389d8f963143508d25e3773cf92e61 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:43.696713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50473ca-41d8-4172-94a2-0cb0dbb7db6e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:43.699642 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:43.796599 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50473ca-41d8-4172-94a2-0cb0dbb7db6e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:43.831086 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a6d5f-1344-4862-ac8a-bd2056692482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:43.831696 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a6d5f-1344-4862-ac8a-bd2056692482 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:43.831972 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a6d5f-1344-4862-ac8a-bd2056692482 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:43.832239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a6d5f-1344-4862-ac8a-bd2056692482 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:43.832968 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a6d5f-1344-4862-ac8a-bd2056692482 None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=09389d8f963143508d25e3773cf92e61, user.id=5cc914b4df9b47eab2298e3538d3c843) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:43.833277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a6d5f-1344-4862-ac8a-bd2056692482 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:43.834730 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a6d5f-1344-4862-ac8a-bd2056692482 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:43.850610 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 575/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:01:43 2022] GET /identity/v3/role_assignments?user.id=5cc914b4df9b47eab2298e3538d3c843&scope.project.id=09389d8f963143508d25e3773cf92e61&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:45.836569 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122d783-0782-4ead-8bbf-687250d0557e None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:45.837093 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122d783-0782-4ead-8bbf-687250d0557e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:45.837210 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122d783-0782-4ead-8bbf-687250d0557e None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:45.838609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 587/1162] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:45.844152 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393fe2d-e65c-4caa-81c2-ae718d1f5de8 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:45.844566 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393fe2d-e65c-4caa-81c2-ae718d1f5de8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:45.844699 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393fe2d-e65c-4caa-81c2-ae718d1f5de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:45.863259 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5393fe2d-e65c-4caa-81c2-ae718d1f5de8 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:45.864436 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5393fe2d-e65c-4caa-81c2-ae718d1f5de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:45.924526 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:45.932515 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 576/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:46.207313 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122d783-0782-4ead-8bbf-687250d0557e None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:46.311337 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122d783-0782-4ead-8bbf-687250d0557e None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:46.335476 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:46.346607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4db196-2452-4c2f-85e4-dcd985a561cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:46.347205 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4db196-2452-4c2f-85e4-dcd985a561cf None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:46.347479 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4db196-2452-4c2f-85e4-dcd985a561cf None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:46.347748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4db196-2452-4c2f-85e4-dcd985a561cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:46.414347 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4db196-2452-4c2f-85e4-dcd985a561cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:46.415814 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4db196-2452-4c2f-85e4-dcd985a561cf None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:46.523086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 588/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:01:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:01:46.529231 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393fe2d-e65c-4caa-81c2-ae718d1f5de8 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:46.628328 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393fe2d-e65c-4caa-81c2-ae718d1f5de8 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:46.663722 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed717e1-1563-4f09-be10-e0086b06aeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:46.664347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed717e1-1563-4f09-be10-e0086b06aeeb None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:46.664620 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed717e1-1563-4f09-be10-e0086b06aeeb None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:46.664885 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed717e1-1563-4f09-be10-e0086b06aeeb None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:46.672288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed717e1-1563-4f09-be10-e0086b06aeeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:46.673737 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed717e1-1563-4f09-be10-e0086b06aeeb None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-8ed717e1-1563-4f09-be10-e0086b06aeeb None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:46.684886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:46.686171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:46.687177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:46.687177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:46.687177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:46.687177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:46.687177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:46.687177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:46.687177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:46.687177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:46.687177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:46.687177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:46.687177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 23 06:01:46.687177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:01:46.687177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 23 06:01:46.687177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:01:46.687177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 577/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 23 06:01:46 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:46.692125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4db196-2452-4c2f-85e4-dcd985a561cf None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:46.721364 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:46.798799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4db196-2452-4c2f-85e4-dcd985a561cf None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:46.833966 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5b597-9d96-4065-a691-8da249bc0813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:46.834602 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5b597-9d96-4065-a691-8da249bc0813 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:46.834880 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5b597-9d96-4065-a691-8da249bc0813 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:46.835147 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5b597-9d96-4065-a691-8da249bc0813 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:46.835779 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5b597-9d96-4065-a691-8da249bc0813 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:46.836079 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5b597-9d96-4065-a691-8da249bc0813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:46.837433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5b597-9d96-4065-a691-8da249bc0813 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:46.849850 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 589/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Aug 23 06:01:46 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:48.767818 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61058f42-e482-4e45-9a87-feaaf9e9d4d0 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:48.768410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61058f42-e482-4e45-9a87-feaaf9e9d4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:48.768410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61058f42-e482-4e45-9a87-feaaf9e9d4d0 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:48.769886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 578/1167] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:48.775006 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9124e-5edd-43ec-a578-ee220cda0fae None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:48.775458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9124e-5edd-43ec-a578-ee220cda0fae None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:48.775565 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9124e-5edd-43ec-a578-ee220cda0fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:48.794227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-bbe9124e-5edd-43ec-a578-ee220cda0fae None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:48.795440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-bbe9124e-5edd-43ec-a578-ee220cda0fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:48.861768 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 590/1168] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:49.130806 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61058f42-e482-4e45-9a87-feaaf9e9d4d0 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:49.191755 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:49.237335 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61058f42-e482-4e45-9a87-feaaf9e9d4d0 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:49.284733 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e60cd-0459-4732-97b0-8bcc381d9991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:49.285329 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e60cd-0459-4732-97b0-8bcc381d9991 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:49.285606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e60cd-0459-4732-97b0-8bcc381d9991 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:49.285876 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e60cd-0459-4732-97b0-8bcc381d9991 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:49.286550 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e60cd-0459-4732-97b0-8bcc381d9991 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:49.287979 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e60cd-0459-4732-97b0-8bcc381d9991 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:49.306798 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 579/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Tue Aug 23 06:01:49 2022] POST /identity/v3/domains => generated 267 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:01:51.272432 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15797698-73fc-4e45-989e-6093d9da61f1 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:51.272996 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15797698-73fc-4e45-989e-6093d9da61f1 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:51.272996 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15797698-73fc-4e45-989e-6093d9da61f1 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:51.274328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 591/1170] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:51.281541 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694e759-c437-432f-a7eb-09986fa71d46 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:51.282045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694e759-c437-432f-a7eb-09986fa71d46 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:51.282159 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694e759-c437-432f-a7eb-09986fa71d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:51.300836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5694e759-c437-432f-a7eb-09986fa71d46 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:51.302033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5694e759-c437-432f-a7eb-09986fa71d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:51.371225 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 580/1171] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:51.644481 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15797698-73fc-4e45-989e-6093d9da61f1 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:51.664085 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:51.747179 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15797698-73fc-4e45-989e-6093d9da61f1 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:51.784223 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a761b2-7d62-48c0-bb22-b4c3c042ac29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:51.784855 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a761b2-7d62-48c0-bb22-b4c3c042ac29 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:51.785132 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a761b2-7d62-48c0-bb22-b4c3c042ac29 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:51.785402 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a761b2-7d62-48c0-bb22-b4c3c042ac29 None admin] PATH_INFO: `/v3/domains/d6fa504ebad14abf91eba465779ed545` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:51.796222 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a761b2-7d62-48c0-bb22-b4c3c042ac29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d6fa504ebad14abf91eba465779ed545)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:51.797667 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a761b2-7d62-48c0-bb22-b4c3c042ac29 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:51.799330 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 592/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Tue Aug 23 06:01:51 2022] GET /identity/v3/domains/d6fa504ebad14abf91eba465779ed545 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:51.805191 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694e759-c437-432f-a7eb-09986fa71d46 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:51.851364 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:51.908724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694e759-c437-432f-a7eb-09986fa71d46 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:51.942025 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec6e60-9910-4761-a06e-26909877b261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:51.942660 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec6e60-9910-4761-a06e-26909877b261 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:51.942940 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec6e60-9910-4761-a06e-26909877b261 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:51.943211 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec6e60-9910-4761-a06e-26909877b261 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:51.943847 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec6e60-9910-4761-a06e-26909877b261 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:51.945281 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec6e60-9910-4761-a06e-26909877b261 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:51.982248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 581/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 23 06:01:51 2022] POST /identity/v3/projects => generated 361 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:01:53.975774 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8233bd-2668-40d7-b378-b2273c7867d1 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:53.976279 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8233bd-2668-40d7-b378-b2273c7867d1 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:53.976379 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8233bd-2668-40d7-b378-b2273c7867d1 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:53.977656 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 593/1174] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:53.983563 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be545399-42ae-4b66-96a4-e458ffd866fd None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:53.983982 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be545399-42ae-4b66-96a4-e458ffd866fd None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:53.984119 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be545399-42ae-4b66-96a4-e458ffd866fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:54.002967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-be545399-42ae-4b66-96a4-e458ffd866fd None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:54.004163 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-be545399-42ae-4b66-96a4-e458ffd866fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:54.081279 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 582/1175] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:01:54.356466 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8233bd-2668-40d7-b378-b2273c7867d1 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:54.436527 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:54.459015 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8233bd-2668-40d7-b378-b2273c7867d1 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:54.492663 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfb8be-098a-4799-98f4-39f0802eb8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:54.493290 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfb8be-098a-4799-98f4-39f0802eb8ff None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:54.493566 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfb8be-098a-4799-98f4-39f0802eb8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:54.493832 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfb8be-098a-4799-98f4-39f0802eb8ff None admin] PATH_INFO: `/v3/domains/d6fa504ebad14abf91eba465779ed545` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:54.495791 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfb8be-098a-4799-98f4-39f0802eb8ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=d6fa504ebad14abf91eba465779ed545)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:54.497218 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfb8be-098a-4799-98f4-39f0802eb8ff None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:54.498996 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 594/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Tue Aug 23 06:01:54 2022] GET /identity/v3/domains/d6fa504ebad14abf91eba465779ed545 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:54.504621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be545399-42ae-4b66-96a4-e458ffd866fd None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:54.532219 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:54.607769 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be545399-42ae-4b66-96a4-e458ffd866fd None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:54.642872 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb05df-4301-4d50-af3c-f62a4a52b5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:54.643471 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb05df-4301-4d50-af3c-f62a4a52b5ff None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:54.643747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb05df-4301-4d50-af3c-f62a4a52b5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:54.644020 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb05df-4301-4d50-af3c-f62a4a52b5ff None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:54.644657 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb05df-4301-4d50-af3c-f62a4a52b5ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:54.646086 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb05df-4301-4d50-af3c-f62a4a52b5ff None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:54.670558 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 583/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 06:01:54 2022] POST /identity/v3/users => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:01:56.634725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8817c8-8d53-4923-980c-ff35359957f0 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:56.635822 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8817c8-8d53-4923-980c-ff35359957f0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:56.635822 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8817c8-8d53-4923-980c-ff35359957f0 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:56.637293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 595/1178] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:01:56.642791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f8097-2939-483d-9bfb-bf2bc4b61469 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:56.643435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f8097-2939-483d-9bfb-bf2bc4b61469 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:56.643435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f8097-2939-483d-9bfb-bf2bc4b61469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:56.669075 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-790f8097-2939-483d-9bfb-bf2bc4b61469 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:01:56.670290 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-790f8097-2939-483d-9bfb-bf2bc4b61469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:01:56.757792 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 584/1179] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:01:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:01:57.038526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8817c8-8d53-4923-980c-ff35359957f0 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:57.145326 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8817c8-8d53-4923-980c-ff35359957f0 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:57.179854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbaf31-fc31-4c5c-92c6-2fe70c871562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:57.180486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbaf31-fc31-4c5c-92c6-2fe70c871562 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:57.180765 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbaf31-fc31-4c5c-92c6-2fe70c871562 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:57.181036 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbaf31-fc31-4c5c-92c6-2fe70c871562 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:57.192640 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cbaf31-fc31-4c5c-92c6-2fe70c871562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-e9cbaf31-fc31-4c5c-92c6-2fe70c871562 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:01:57.197278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:01:57.198842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:01:57.200056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:01:57.200056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:01:57.200056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:01:57.200056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:01:57.200056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:01:57.200056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:01:57.200056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:01:57.200056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:01:57.200056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:01:57.200056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:01:57.200056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:01:57.200056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:01:57.200056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 596/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:01:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:01:57.205731 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f8097-2939-483d-9bfb-bf2bc4b61469 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:57.221440 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:57.315036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f8097-2939-483d-9bfb-bf2bc4b61469 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:57.351495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c3604-1337-496a-8724-540701fd88a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:57.352102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c3604-1337-496a-8724-540701fd88a2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:57.352379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c3604-1337-496a-8724-540701fd88a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:57.352652 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c3604-1337-496a-8724-540701fd88a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:57.353319 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c3604-1337-496a-8724-540701fd88a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:57.353619 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c3604-1337-496a-8724-540701fd88a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:57.355286 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c3604-1337-496a-8724-540701fd88a2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:57.364348 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 585/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:01:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:57.370300 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbaf31-fc31-4c5c-92c6-2fe70c871562 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:57.379586 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:57.473018 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbaf31-fc31-4c5c-92c6-2fe70c871562 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:57.518348 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a4738-4794-4814-94c3-b94ff0780e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:57.519183 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a4738-4794-4814-94c3-b94ff0780e01 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:57.519465 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a4738-4794-4814-94c3-b94ff0780e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:57.519737 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a4738-4794-4814-94c3-b94ff0780e01 None admin] PATH_INFO: `/v3/users/924a3a0ae9564ab78593620eff11da9e` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:57.535238 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a4738-4794-4814-94c3-b94ff0780e01 None admin] RBAC: Authorizing `identity:get_user(user_id=924a3a0ae9564ab78593620eff11da9e)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:57.536723 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a4738-4794-4814-94c3-b94ff0780e01 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:57.538490 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 597/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 06:01:57 2022] GET /identity/v3/users/924a3a0ae9564ab78593620eff11da9e => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:57.545438 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c3604-1337-496a-8724-540701fd88a2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:57.629534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:57.653975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c3604-1337-496a-8724-540701fd88a2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:57.688248 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bff4a-7c8e-4fbc-877e-3c0ac609774d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:57.688848 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bff4a-7c8e-4fbc-877e-3c0ac609774d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:57.689126 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bff4a-7c8e-4fbc-877e-3c0ac609774d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:57.689395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bff4a-7c8e-4fbc-877e-3c0ac609774d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:57.758854 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bff4a-7c8e-4fbc-877e-3c0ac609774d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:57.760923 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bff4a-7c8e-4fbc-877e-3c0ac609774d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:57.857281 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:57.889789 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 586/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:01:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:01:57.896025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a4738-4794-4814-94c3-b94ff0780e01 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:57.974658 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:01:58.022805 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a4738-4794-4814-94c3-b94ff0780e01 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:58.070486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81015d2b-f6e4-480c-afc6-a06414e4191c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:58.071133 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81015d2b-f6e4-480c-afc6-a06414e4191c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:58.071410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81015d2b-f6e4-480c-afc6-a06414e4191c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:58.071717 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81015d2b-f6e4-480c-afc6-a06414e4191c None admin] PATH_INFO: `/v3/projects/ae05402c408745bb810c85d86c985896` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:58.074067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81015d2b-f6e4-480c-afc6-a06414e4191c None admin] RBAC: Authorizing `identity:get_project(project_id=ae05402c408745bb810c85d86c985896)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:58.075601 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81015d2b-f6e4-480c-afc6-a06414e4191c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:58.078019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 598/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 06:01:57 2022] GET /identity/v3/projects/ae05402c408745bb810c85d86c985896 => generated 361 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:01:58.085474 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bff4a-7c8e-4fbc-877e-3c0ac609774d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:01:58.217106 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bff4a-7c8e-4fbc-877e-3c0ac609774d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:01:58.253843 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851559c-ceef-4cb8-9700-f62b5fb71f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:01:58.254440 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851559c-ceef-4cb8-9700-f62b5fb71f10 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:01:58.254771 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851559c-ceef-4cb8-9700-f62b5fb71f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:01:58.255047 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851559c-ceef-4cb8-9700-f62b5fb71f10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:01:58.255785 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851559c-ceef-4cb8-9700-f62b5fb71f10 None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=ae05402c408745bb810c85d86c985896, user.id=924a3a0ae9564ab78593620eff11da9e) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:01:58.256088 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851559c-ceef-4cb8-9700-f62b5fb71f10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:01:58.257617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851559c-ceef-4cb8-9700-f62b5fb71f10 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:01:58.264748 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 587/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:01:58 2022] GET /identity/v3/role_assignments?user.id=924a3a0ae9564ab78593620eff11da9e&scope.project.id=ae05402c408745bb810c85d86c985896&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:02:00.226608 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ff35d-8c93-4a54-b0bc-be1444425edd None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:02:00.227113 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ff35d-8c93-4a54-b0bc-be1444425edd None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:02:00.227113 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ff35d-8c93-4a54-b0bc-be1444425edd None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:02:00.228460 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 599/1186] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:02:00.234825 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8225843-aabf-47b1-a4c8-59f969a1a756 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:02:00.235341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8225843-aabf-47b1-a4c8-59f969a1a756 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:02:00.235341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8225843-aabf-47b1-a4c8-59f969a1a756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:02:00.254756 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-f8225843-aabf-47b1-a4c8-59f969a1a756 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:02:00.255966 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f8225843-aabf-47b1-a4c8-59f969a1a756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:02:00.290299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:02:00.326042 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 588/1187] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:02:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:02:00.592470 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ff35d-8c93-4a54-b0bc-be1444425edd None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:02:00.702925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ff35d-8c93-4a54-b0bc-be1444425edd None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:02:00.733110 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:02:00.739020 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700534c-d03b-4e5a-8480-9122e0805c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:02:00.739657 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700534c-d03b-4e5a-8480-9122e0805c3b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:02:00.739935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700534c-d03b-4e5a-8480-9122e0805c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:02:00.740208 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700534c-d03b-4e5a-8480-9122e0805c3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:02:00.751704 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700534c-d03b-4e5a-8480-9122e0805c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-7700534c-d03b-4e5a-8480-9122e0805c3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:02:00.756463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:02:00.757435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:02:00.758301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:02:00.758301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:02:00.758301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:02:00.758301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:02:00.758301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:02:00.758301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:02:00.758301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:02:00.758301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:02:00.758301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:02:00.758301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:02:00.758301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:02:00.758301 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:02:00.759258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 600/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:02:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:02:00.764691 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8225843-aabf-47b1-a4c8-59f969a1a756 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:02:00.876866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8225843-aabf-47b1-a4c8-59f969a1a756 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:02:00.912066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81995685-b55e-4645-be05-919aec0ec73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:02:00.912671 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81995685-b55e-4645-be05-919aec0ec73e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:02:00.912949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81995685-b55e-4645-be05-919aec0ec73e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:02:00.913219 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81995685-b55e-4645-be05-919aec0ec73e None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:02:00.913883 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81995685-b55e-4645-be05-919aec0ec73e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:02:00.914182 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81995685-b55e-4645-be05-919aec0ec73e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:02:00.915588 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81995685-b55e-4645-be05-919aec0ec73e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:02:00.924328 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 589/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:02:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:02:00.930223 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700534c-d03b-4e5a-8480-9122e0805c3b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:02:01.041216 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700534c-d03b-4e5a-8480-9122e0805c3b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:02:01.075890 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce392519-42db-463c-a667-d280d67e755d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:02:01.076508 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce392519-42db-463c-a667-d280d67e755d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:02:01.076793 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce392519-42db-463c-a667-d280d67e755d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:02:01.077066 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce392519-42db-463c-a667-d280d67e755d None admin] PATH_INFO: `/v3/users/924a3a0ae9564ab78593620eff11da9e` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:02:01.078189 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce392519-42db-463c-a667-d280d67e755d None admin] RBAC: Authorizing `identity:get_user(user_id=924a3a0ae9564ab78593620eff11da9e)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:02:01.079652 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce392519-42db-463c-a667-d280d67e755d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:02:01.081244 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 601/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 06:02:00 2022] GET /identity/v3/users/924a3a0ae9564ab78593620eff11da9e => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:02:01.087204 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81995685-b55e-4645-be05-919aec0ec73e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:02:01.119112 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:02:01.193021 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81995685-b55e-4645-be05-919aec0ec73e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:02:01.227994 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ec74b-e891-41ae-b572-ca15dd539ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:02:01.228621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ec74b-e891-41ae-b572-ca15dd539ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:02:01.228954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ec74b-e891-41ae-b572-ca15dd539ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:02:01.229370 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ec74b-e891-41ae-b572-ca15dd539ff8 None admin] PATH_INFO: `/v3/projects/ae05402c408745bb810c85d86c985896` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:02:01.230725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ec74b-e891-41ae-b572-ca15dd539ff8 None admin] RBAC: Authorizing `identity:get_project(project_id=ae05402c408745bb810c85d86c985896)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:02:01.232166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ec74b-e891-41ae-b572-ca15dd539ff8 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:02:01.233816 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 590/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 06:02:01 2022] GET /identity/v3/projects/ae05402c408745bb810c85d86c985896 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:02:01.240044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce392519-42db-463c-a667-d280d67e755d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:02:01.301887 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:02:01.344222 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce392519-42db-463c-a667-d280d67e755d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:02:01.378644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73defcb-52c2-4ae8-bbff-a49389b77274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:02:01.379311 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73defcb-52c2-4ae8-bbff-a49389b77274 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:02:01.379590 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73defcb-52c2-4ae8-bbff-a49389b77274 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:02:01.379860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73defcb-52c2-4ae8-bbff-a49389b77274 None admin] PATH_INFO: `/v3/projects/ae05402c408745bb810c85d86c985896/users/924a3a0ae9564ab78593620eff11da9e/roles/f6acb6fdca5a44de9ba00b2b0d4d3202` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:02:01.381776 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73defcb-52c2-4ae8-bbff-a49389b77274 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae05402c408745bb810c85d86c985896, user_id=924a3a0ae9564ab78593620eff11da9e, role_id=f6acb6fdca5a44de9ba00b2b0d4d3202)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:02:01.383253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73defcb-52c2-4ae8-bbff-a49389b77274 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:02:01.397890 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 602/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:02:01 2022] PUT /identity/v3/projects/ae05402c408745bb810c85d86c985896/users/924a3a0ae9564ab78593620eff11da9e/roles/f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:02:03.312446 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0e6e8-33b4-4aad-9c0e-7c3020563b2a None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:02:03.312999 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0e6e8-33b4-4aad-9c0e-7c3020563b2a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:02:03.312999 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0e6e8-33b4-4aad-9c0e-7c3020563b2a None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:02:03.314323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 591/1193] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:02:03.319855 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b6963-29d4-414c-9c7b-1b5e89b91332 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:02:03.320251 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b6963-29d4-414c-9c7b-1b5e89b91332 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:02:03.320402 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b6963-29d4-414c-9c7b-1b5e89b91332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:02:03.338428 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-f20b6963-29d4-414c-9c7b-1b5e89b91332 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:02:03.339660 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f20b6963-29d4-414c-9c7b-1b5e89b91332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:02:03.406959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 603/1194] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:02:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:02:03.672175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0e6e8-33b4-4aad-9c0e-7c3020563b2a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:02:03.776212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:02:03.781323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0e6e8-33b4-4aad-9c0e-7c3020563b2a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:02:03.817862 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d341add-6bfc-4afd-8ff6-2e8cdfa9efa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:02:03.818489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d341add-6bfc-4afd-8ff6-2e8cdfa9efa6 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:02:03.818794 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d341add-6bfc-4afd-8ff6-2e8cdfa9efa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:02:03.819068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d341add-6bfc-4afd-8ff6-2e8cdfa9efa6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:02:03.829783 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d341add-6bfc-4afd-8ff6-2e8cdfa9efa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-5d341add-6bfc-4afd-8ff6-2e8cdfa9efa6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:02:03.834606 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:02:03.835679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:02:03.836524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:02:03.836524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:02:03.836524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:02:03.836524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:02:03.836524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:02:03.836524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:02:03.836524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:02:03.836524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:02:03.836524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:02:03.836524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:02:03.836524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:02:03.836524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:02:03.836524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 592/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:02:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:02:03.842782 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b6963-29d4-414c-9c7b-1b5e89b91332 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:02:03.892678 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:02:03.943702 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b6963-29d4-414c-9c7b-1b5e89b91332 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:02:03.980727 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ed73f-c621-4079-beb0-ce0a981c9013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:02:03.981330 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ed73f-c621-4079-beb0-ce0a981c9013 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:02:03.981608 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ed73f-c621-4079-beb0-ce0a981c9013 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:02:03.981879 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ed73f-c621-4079-beb0-ce0a981c9013 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:02:03.982562 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ed73f-c621-4079-beb0-ce0a981c9013 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:02:03.982868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ed73f-c621-4079-beb0-ce0a981c9013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:02:03.984250 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ed73f-c621-4079-beb0-ce0a981c9013 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:02:03.992911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 604/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:02:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:02:03.999307 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d341add-6bfc-4afd-8ff6-2e8cdfa9efa6 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:02:04.101963 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d341add-6bfc-4afd-8ff6-2e8cdfa9efa6 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:02:04.134866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439a26b-24a9-4731-ab9c-5b86f7865021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:02:04.135999 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439a26b-24a9-4731-ab9c-5b86f7865021 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:02:04.136273 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439a26b-24a9-4731-ab9c-5b86f7865021 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:02:04.136540 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439a26b-24a9-4731-ab9c-5b86f7865021 None admin] PATH_INFO: `/v3/users/924a3a0ae9564ab78593620eff11da9e` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:02:04.137647 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439a26b-24a9-4731-ab9c-5b86f7865021 None admin] RBAC: Authorizing `identity:get_user(user_id=924a3a0ae9564ab78593620eff11da9e)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:02:04.139124 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439a26b-24a9-4731-ab9c-5b86f7865021 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:02:04.140721 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 593/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Aug 23 06:02:03 2022] GET /identity/v3/users/924a3a0ae9564ab78593620eff11da9e => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:02:04.146728 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ed73f-c621-4079-beb0-ce0a981c9013 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:02:04.250808 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ed73f-c621-4079-beb0-ce0a981c9013 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:02:04.264568 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:02:04.286800 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00853657-406e-4fc3-a42b-82ae03277da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:02:04.287411 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00853657-406e-4fc3-a42b-82ae03277da7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:02:04.287688 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00853657-406e-4fc3-a42b-82ae03277da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:02:04.287958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00853657-406e-4fc3-a42b-82ae03277da7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:02:04.355854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00853657-406e-4fc3-a42b-82ae03277da7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:02:04.357306 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00853657-406e-4fc3-a42b-82ae03277da7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:02:04.455463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 605/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:02:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:02:04.461410 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439a26b-24a9-4731-ab9c-5b86f7865021 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:02:04.491547 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:02:04.563697 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439a26b-24a9-4731-ab9c-5b86f7865021 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:02:04.598833 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb4e03-7146-4d24-b41b-96d4b903972c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:02:04.599470 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb4e03-7146-4d24-b41b-96d4b903972c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:02:04.599746 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb4e03-7146-4d24-b41b-96d4b903972c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:02:04.600017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb4e03-7146-4d24-b41b-96d4b903972c None admin] PATH_INFO: `/v3/projects/ae05402c408745bb810c85d86c985896` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:02:04.601176 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb4e03-7146-4d24-b41b-96d4b903972c None admin] RBAC: Authorizing `identity:get_project(project_id=ae05402c408745bb810c85d86c985896)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:02:04.602662 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb4e03-7146-4d24-b41b-96d4b903972c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:02:04.604587 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 594/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Aug 23 06:02:04 2022] GET /identity/v3/projects/ae05402c408745bb810c85d86c985896 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:02:04.611836 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00853657-406e-4fc3-a42b-82ae03277da7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:02:04.627015 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:02:04.713026 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00853657-406e-4fc3-a42b-82ae03277da7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:02:04.755758 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85c586-f73f-42dc-9b73-40c4db16bda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:02:04.756355 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85c586-f73f-42dc-9b73-40c4db16bda8 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:02:04.756631 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85c586-f73f-42dc-9b73-40c4db16bda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:02:04.756896 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85c586-f73f-42dc-9b73-40c4db16bda8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:02:04.757621 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85c586-f73f-42dc-9b73-40c4db16bda8 None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=ae05402c408745bb810c85d86c985896, user.id=924a3a0ae9564ab78593620eff11da9e) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:02:04.757921 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85c586-f73f-42dc-9b73-40c4db16bda8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:02:04.759524 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85c586-f73f-42dc-9b73-40c4db16bda8 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:02:04.765610 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 606/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:02:04 2022] GET /identity/v3/role_assignments?user.id=924a3a0ae9564ab78593620eff11da9e&scope.project.id=ae05402c408745bb810c85d86c985896&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:02:14.595089 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=52 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:02:14.762915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=62 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:04.298019 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf5f02-bc8d-43a9-b626-ad4fd17c0fee None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:04.298855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf5f02-bc8d-43a9-b626-ad4fd17c0fee None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:04.298855 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf5f02-bc8d-43a9-b626-ad4fd17c0fee None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:04.304565 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 595/1201] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:03:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:03:04.306990 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8021695-1e2a-4267-9e26-472daaf74053 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:04.307713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8021695-1e2a-4267-9e26-472daaf74053 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:04.307713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8021695-1e2a-4267-9e26-472daaf74053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:04.329908 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-f8021695-1e2a-4267-9e26-472daaf74053 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:03:04.331130 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f8021695-1e2a-4267-9e26-472daaf74053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:03:04.430981 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 607/1202] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:03:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:03:04.707523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf5f02-bc8d-43a9-b626-ad4fd17c0fee None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:04.890336 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf5f02-bc8d-43a9-b626-ad4fd17c0fee None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:04.961647 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc2b52-7e70-4d8f-882c-fbb8a56026bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:04.962587 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc2b52-7e70-4d8f-882c-fbb8a56026bd None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:04.963103 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc2b52-7e70-4d8f-882c-fbb8a56026bd None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:04.963650 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc2b52-7e70-4d8f-882c-fbb8a56026bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:04.982092 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc2b52-7e70-4d8f-882c-fbb8a56026bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:04.983987 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc2b52-7e70-4d8f-882c-fbb8a56026bd None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-7ecc2b52-7e70-4d8f-882c-fbb8a56026bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:03:05.018572 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:05.019724 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:03:05.023689 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:03:05.023689 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:05.023689 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:05.023689 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 596/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:05.026864 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8021695-1e2a-4267-9e26-472daaf74053 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:05.160822 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8021695-1e2a-4267-9e26-472daaf74053 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:05.196999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:05.210834 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff4756-2e4f-4470-978f-7f705082be04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:05.217101 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff4756-2e4f-4470-978f-7f705082be04 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:05.217399 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff4756-2e4f-4470-978f-7f705082be04 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:05.217710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff4756-2e4f-4470-978f-7f705082be04 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:05.218461 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff4756-2e4f-4470-978f-7f705082be04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:05.218821 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff4756-2e4f-4470-978f-7f705082be04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:05.220293 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff4756-2e4f-4470-978f-7f705082be04 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:05.238572 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 608/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:05.246398 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc2b52-7e70-4d8f-882c-fbb8a56026bd None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:05.330873 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:05.416581 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc2b52-7e70-4d8f-882c-fbb8a56026bd None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:05.467042 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee6e01-7da6-41c4-ace7-325ea6f245f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:05.467587 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee6e01-7da6-41c4-ace7-325ea6f245f2 None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:05.467865 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee6e01-7da6-41c4-ace7-325ea6f245f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:05.468134 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee6e01-7da6-41c4-ace7-325ea6f245f2 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:05.468780 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee6e01-7da6-41c4-ace7-325ea6f245f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:05.470219 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee6e01-7da6-41c4-ace7-325ea6f245f2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:05.510544 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 597/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Aug 23 06:03:05 2022] POST /identity/v3/users => generated 251 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:03:07.885949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63eec5-9007-43e9-8ac1-52a35f5b5491 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:07.886506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63eec5-9007-43e9-8ac1-52a35f5b5491 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:07.886506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63eec5-9007-43e9-8ac1-52a35f5b5491 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:07.889431 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 609/1206] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:03:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:03:07.894745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796276c8-38a6-4178-a2c7-64383aa287eb None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:07.895266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796276c8-38a6-4178-a2c7-64383aa287eb None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:07.895358 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796276c8-38a6-4178-a2c7-64383aa287eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:07.917344 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-796276c8-38a6-4178-a2c7-64383aa287eb None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:03:07.918555 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-796276c8-38a6-4178-a2c7-64383aa287eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:03:08.009218 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 598/1207] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:03:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:03:08.340157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63eec5-9007-43e9-8ac1-52a35f5b5491 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:08.506764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63eec5-9007-43e9-8ac1-52a35f5b5491 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:08.552749 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266765f3-5002-433d-ada5-771e5a1c275f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:08.554437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266765f3-5002-433d-ada5-771e5a1c275f None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:08.555170 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266765f3-5002-433d-ada5-771e5a1c275f None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:08.556004 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266765f3-5002-433d-ada5-771e5a1c275f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:08.576507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266765f3-5002-433d-ada5-771e5a1c275f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-266765f3-5002-433d-ada5-771e5a1c275f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:03:08.585611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:08.586694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:08.588863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:08.588863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:08.588863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:08.588863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:08.588863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:03:08.588863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:03:08.588863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:03:08.588863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:03:08.588863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:03:08.588863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:03:08.588863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:03:08.588863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:08.588863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 610/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:03:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:08.599945 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796276c8-38a6-4178-a2c7-64383aa287eb None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:08.658423 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:08.752974 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796276c8-38a6-4178-a2c7-64383aa287eb None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:08.808975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7385a-44eb-406e-ba6b-e3c0e29eed4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:08.809609 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7385a-44eb-406e-ba6b-e3c0e29eed4c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:08.809948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7385a-44eb-406e-ba6b-e3c0e29eed4c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:08.810186 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7385a-44eb-406e-ba6b-e3c0e29eed4c None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:08.811001 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7385a-44eb-406e-ba6b-e3c0e29eed4c None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:08.811302 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7385a-44eb-406e-ba6b-e3c0e29eed4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:08.812886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7385a-44eb-406e-ba6b-e3c0e29eed4c None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:08.826680 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 599/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:03:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:08.831140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266765f3-5002-433d-ada5-771e5a1c275f None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:08.892504 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:08.956536 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266765f3-5002-433d-ada5-771e5a1c275f None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:09.000817 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72578e-dcdc-4a70-94db-ee3bb61b45ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:09.001275 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72578e-dcdc-4a70-94db-ee3bb61b45ff None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:09.001606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72578e-dcdc-4a70-94db-ee3bb61b45ff None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:09.001813 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72578e-dcdc-4a70-94db-ee3bb61b45ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:09.020885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72578e-dcdc-4a70-94db-ee3bb61b45ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:09.022366 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72578e-dcdc-4a70-94db-ee3bb61b45ff None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-8a72578e-dcdc-4a70-94db-ee3bb61b45ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:03:09.042913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:09.045693 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:03:09.046827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:03:09.046827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:09.046827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:09.046827 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 611/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:09.050550 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7385a-44eb-406e-ba6b-e3c0e29eed4c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:09.178878 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:09.183489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7385a-44eb-406e-ba6b-e3c0e29eed4c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:09.234686 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474fa2a-992e-47bd-ad8f-cb0529438541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:09.235416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474fa2a-992e-47bd-ad8f-cb0529438541 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:09.235702 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474fa2a-992e-47bd-ad8f-cb0529438541 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:09.236007 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474fa2a-992e-47bd-ad8f-cb0529438541 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:09.236773 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474fa2a-992e-47bd-ad8f-cb0529438541 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:09.237218 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474fa2a-992e-47bd-ad8f-cb0529438541 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:09.238813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474fa2a-992e-47bd-ad8f-cb0529438541 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:09.256356 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 600/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:09.263158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72578e-dcdc-4a70-94db-ee3bb61b45ff None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:09.413104 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72578e-dcdc-4a70-94db-ee3bb61b45ff None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:09.415102 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:09.476066 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e41f3-96d8-43f8-a92f-b98d225145b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:09.477470 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e41f3-96d8-43f8-a92f-b98d225145b4 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:09.478494 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e41f3-96d8-43f8-a92f-b98d225145b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:09.479139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e41f3-96d8-43f8-a92f-b98d225145b4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:09.489124 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e41f3-96d8-43f8-a92f-b98d225145b4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:09.491098 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e41f3-96d8-43f8-a92f-b98d225145b4 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-2e5e41f3-96d8-43f8-a92f-b98d225145b4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:03:09.506437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:09.507637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:09.508592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:09.508592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:09.508592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:03:09.508592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:03:09.508592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:03:09.508592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:03:09.508592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:03:09.508592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:03:09.508592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 06:03:09.508592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:09.510860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 612/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:09.518089 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474fa2a-992e-47bd-ad8f-cb0529438541 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:09.652536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474fa2a-992e-47bd-ad8f-cb0529438541 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:09.693699 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50c7f2-5c41-4f8d-9a27-fd04e2ceed9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:09.694850 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50c7f2-5c41-4f8d-9a27-fd04e2ceed9b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:09.695394 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50c7f2-5c41-4f8d-9a27-fd04e2ceed9b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:09.695885 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50c7f2-5c41-4f8d-9a27-fd04e2ceed9b None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:09.696989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50c7f2-5c41-4f8d-9a27-fd04e2ceed9b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:09.697541 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50c7f2-5c41-4f8d-9a27-fd04e2ceed9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:09.699699 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50c7f2-5c41-4f8d-9a27-fd04e2ceed9b None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:09.725658 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 601/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 23 06:03:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:09.733426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e41f3-96d8-43f8-a92f-b98d225145b4 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:09.862909 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e41f3-96d8-43f8-a92f-b98d225145b4 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:09.904769 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f8972-9481-4201-af66-e9d732980940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:09.905466 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f8972-9481-4201-af66-e9d732980940 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:09.905712 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f8972-9481-4201-af66-e9d732980940 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:09.906138 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f8972-9481-4201-af66-e9d732980940 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:09.945356 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:09.996944 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f8972-9481-4201-af66-e9d732980940 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:09.998646 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f8972-9481-4201-af66-e9d732980940 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:10.126618 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50c7f2-5c41-4f8d-9a27-fd04e2ceed9b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:10.127168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 613/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:03:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:03:10.160833 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:10.271039 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50c7f2-5c41-4f8d-9a27-fd04e2ceed9b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:10.320654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce2fb9-eec3-4223-b3d9-f68145d7cc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:10.321416 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce2fb9-eec3-4223-b3d9-f68145d7cc33 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:10.321845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce2fb9-eec3-4223-b3d9-f68145d7cc33 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:10.322311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce2fb9-eec3-4223-b3d9-f68145d7cc33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:10.334461 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce2fb9-eec3-4223-b3d9-f68145d7cc33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:10.335972 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce2fb9-eec3-4223-b3d9-f68145d7cc33 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-9bce2fb9-eec3-4223-b3d9-f68145d7cc33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:03:10.354934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:10.355997 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:03:10.356963 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:03:10.356963 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:10.356963 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:10.356963 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 602/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:10.361937 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f8972-9481-4201-af66-e9d732980940 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:10.446772 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:10.504911 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f8972-9481-4201-af66-e9d732980940 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:10.557752 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607f7f5-2a09-4ce9-a2be-1a1adb4847db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:10.558814 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607f7f5-2a09-4ce9-a2be-1a1adb4847db None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:10.559323 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607f7f5-2a09-4ce9-a2be-1a1adb4847db None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:10.559847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607f7f5-2a09-4ce9-a2be-1a1adb4847db None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:10.560763 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607f7f5-2a09-4ce9-a2be-1a1adb4847db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:10.561495 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607f7f5-2a09-4ce9-a2be-1a1adb4847db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:10.563123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607f7f5-2a09-4ce9-a2be-1a1adb4847db None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:10.585194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 614/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:10.586435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce2fb9-eec3-4223-b3d9-f68145d7cc33 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:10.699414 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:10.751254 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce2fb9-eec3-4223-b3d9-f68145d7cc33 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:10.794097 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc31b1-7686-43b5-bdbf-6c5632b3111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:10.794911 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc31b1-7686-43b5-bdbf-6c5632b3111e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:10.795300 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc31b1-7686-43b5-bdbf-6c5632b3111e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:10.795677 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc31b1-7686-43b5-bdbf-6c5632b3111e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:10.803668 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc31b1-7686-43b5-bdbf-6c5632b3111e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:10.805687 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc31b1-7686-43b5-bdbf-6c5632b3111e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:10.820615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-6ddc31b1-7686-43b5-bdbf-6c5632b3111e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:03:10.820615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:10.820615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:10.820615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:10.820615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:10.820615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:10.820615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:10.820615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:10.820615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:10.820615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:10.820615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:10.820615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:10.820615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:03:10.820615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:03:10.820615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:10.821822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:10.822972 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:03:10.822972 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:03:10.822972 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:03:10.822972 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:03:10.822972 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:03:10.822972 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:03:10.822972 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:03:10.822972 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:10.825115 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 603/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:03:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:10.831457 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607f7f5-2a09-4ce9-a2be-1a1adb4847db None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:10.967942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607f7f5-2a09-4ce9-a2be-1a1adb4847db None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:10.998748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:11.013286 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b826563-7ed1-43ee-95f2-fb406a562894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:11.013896 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b826563-7ed1-43ee-95f2-fb406a562894 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:11.014186 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b826563-7ed1-43ee-95f2-fb406a562894 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:11.014439 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b826563-7ed1-43ee-95f2-fb406a562894 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:11.015174 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b826563-7ed1-43ee-95f2-fb406a562894 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:11.015476 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b826563-7ed1-43ee-95f2-fb406a562894 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:11.016903 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b826563-7ed1-43ee-95f2-fb406a562894 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:11.030835 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 615/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:03:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:03:11.037313 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc31b1-7686-43b5-bdbf-6c5632b3111e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:11.156072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc31b1-7686-43b5-bdbf-6c5632b3111e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:11.187152 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:11.195312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538cf68-63d0-403c-86e3-ea2a7c0825f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:11.195926 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538cf68-63d0-403c-86e3-ea2a7c0825f5 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:11.196201 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538cf68-63d0-403c-86e3-ea2a7c0825f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:11.197654 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538cf68-63d0-403c-86e3-ea2a7c0825f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:11.198407 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538cf68-63d0-403c-86e3-ea2a7c0825f5 None admin] RBAC: Adding query filter params (role.id=3209a3d044b84a0ebd7cf1edf2da7120, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=0e16f51a2c714b2fb3d7875b589f9a3c) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:11.198763 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538cf68-63d0-403c-86e3-ea2a7c0825f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:11.200202 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538cf68-63d0-403c-86e3-ea2a7c0825f5 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:11.206873 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 604/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:03:11 2022] GET /identity/v3/role_assignments?user.id=0e16f51a2c714b2fb3d7875b589f9a3c&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=3209a3d044b84a0ebd7cf1edf2da7120 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:13.432940 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8c1e4-0414-4621-a3c8-5a6b78cfd36a None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:13.433557 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8c1e4-0414-4621-a3c8-5a6b78cfd36a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:13.433557 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8c1e4-0414-4621-a3c8-5a6b78cfd36a None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:13.435030 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 616/1220] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:03:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:03:13.440279 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914196e-1ce4-4665-9a5e-370bc6f8f2db None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:13.440785 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914196e-1ce4-4665-9a5e-370bc6f8f2db None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:13.440896 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914196e-1ce4-4665-9a5e-370bc6f8f2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:13.462914 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-7914196e-1ce4-4665-9a5e-370bc6f8f2db None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:03:13.464328 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7914196e-1ce4-4665-9a5e-370bc6f8f2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:03:13.546750 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 605/1221] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:03:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:03:13.816772 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8c1e4-0414-4621-a3c8-5a6b78cfd36a None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:13.937525 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8c1e4-0414-4621-a3c8-5a6b78cfd36a None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:13.983191 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee0146-fe30-40d9-8fcb-e4d05b6332e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:13.984012 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee0146-fe30-40d9-8fcb-e4d05b6332e0 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:13.984310 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee0146-fe30-40d9-8fcb-e4d05b6332e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:13.984590 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee0146-fe30-40d9-8fcb-e4d05b6332e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:13.999313 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee0146-fe30-40d9-8fcb-e4d05b6332e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-b7ee0146-fe30-40d9-8fcb-e4d05b6332e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:03:14.004506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:14.005526 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:14.006604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:14.006604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:14.006604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:14.006604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:14.006604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:03:14.006604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:03:14.006604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:03:14.006604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:03:14.006604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:03:14.006604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:03:14.006604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:03:14.006604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:14.006604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 617/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:03:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:14.012135 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914196e-1ce4-4665-9a5e-370bc6f8f2db None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:14.142915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914196e-1ce4-4665-9a5e-370bc6f8f2db None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:14.169187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:14.184948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55ec2b-beb4-4c65-bf81-11fbb8f36da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:14.185598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55ec2b-beb4-4c65-bf81-11fbb8f36da1 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:14.185908 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55ec2b-beb4-4c65-bf81-11fbb8f36da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:14.186254 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55ec2b-beb4-4c65-bf81-11fbb8f36da1 None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:14.187259 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55ec2b-beb4-4c65-bf81-11fbb8f36da1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:14.187580 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55ec2b-beb4-4c65-bf81-11fbb8f36da1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:14.189279 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55ec2b-beb4-4c65-bf81-11fbb8f36da1 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:14.200493 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 606/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:03:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:03:14.211592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee0146-fe30-40d9-8fcb-e4d05b6332e0 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:14.273367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:14.362103 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee0146-fe30-40d9-8fcb-e4d05b6332e0 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:14.405477 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da447576-b235-4b97-8acc-a6ae28733942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:14.406065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da447576-b235-4b97-8acc-a6ae28733942 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:14.406364 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da447576-b235-4b97-8acc-a6ae28733942 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:14.406746 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da447576-b235-4b97-8acc-a6ae28733942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:14.417960 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da447576-b235-4b97-8acc-a6ae28733942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:14.419541 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da447576-b235-4b97-8acc-a6ae28733942 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-da447576-b235-4b97-8acc-a6ae28733942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:03:14.436406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:14.437868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:03:14.439467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:03:14.439467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:14.439467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:14.439467 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 618/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:14.444178 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55ec2b-beb4-4c65-bf81-11fbb8f36da1 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:14.576184 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55ec2b-beb4-4c65-bf81-11fbb8f36da1 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:14.614251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa174cf-99fb-4648-8024-6be23821ac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:14.615258 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa174cf-99fb-4648-8024-6be23821ac4a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:14.615560 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa174cf-99fb-4648-8024-6be23821ac4a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:14.615867 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa174cf-99fb-4648-8024-6be23821ac4a None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:14.616508 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa174cf-99fb-4648-8024-6be23821ac4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:14.616821 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa174cf-99fb-4648-8024-6be23821ac4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:14.618291 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa174cf-99fb-4648-8024-6be23821ac4a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:14.633615 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 607/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:14.641099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da447576-b235-4b97-8acc-a6ae28733942 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:14.753437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:14.765042 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da447576-b235-4b97-8acc-a6ae28733942 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:14.804934 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f4724-ab69-4fa9-a197-19e11349f743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:14.805719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f4724-ab69-4fa9-a197-19e11349f743 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:14.806214 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f4724-ab69-4fa9-a197-19e11349f743 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:14.806849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f4724-ab69-4fa9-a197-19e11349f743 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:14.815631 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f4724-ab69-4fa9-a197-19e11349f743 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:14.817280 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f4724-ab69-4fa9-a197-19e11349f743 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-987f4724-ab69-4fa9-a197-19e11349f743 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:03:14.830875 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:14.831867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:14.832798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:14.832798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:14.832798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:03:14.832798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:03:14.832798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:03:14.832798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:03:14.832798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:03:14.832798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:03:14.832798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 06:03:14.832798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:14.832798 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 619/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:03:14.838841 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa174cf-99fb-4648-8024-6be23821ac4a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:14.909722 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:14.984917 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa174cf-99fb-4648-8024-6be23821ac4a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:15.022868 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ed84a-080e-44a9-a157-4cd8468975a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:15.023402 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ed84a-080e-44a9-a157-4cd8468975a4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:15.023938 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ed84a-080e-44a9-a157-4cd8468975a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:15.023938 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ed84a-080e-44a9-a157-4cd8468975a4 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:15.024660 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ed84a-080e-44a9-a157-4cd8468975a4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:15.024985 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ed84a-080e-44a9-a157-4cd8468975a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:15.026411 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ed84a-080e-44a9-a157-4cd8468975a4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:15.046907 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 608/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 23 06:03:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:03:15.052499 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f4724-ab69-4fa9-a197-19e11349f743 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:15.176288 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f4724-ab69-4fa9-a197-19e11349f743 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:15.213935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7a7d1-e0e2-4080-a4c6-2e1ba814872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:15.214638 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7a7d1-e0e2-4080-a4c6-2e1ba814872d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:15.214967 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7a7d1-e0e2-4080-a4c6-2e1ba814872d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:15.215236 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7a7d1-e0e2-4080-a4c6-2e1ba814872d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:15.231232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7a7d1-e0e2-4080-a4c6-2e1ba814872d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:15.232841 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7a7d1-e0e2-4080-a4c6-2e1ba814872d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-41d7a7d1-e0e2-4080-a4c6-2e1ba814872d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:03:15.252351 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:15.253578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:03:15.260268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:03:15.260268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:15.260268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:15.260268 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 620/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:15.261706 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ed84a-080e-44a9-a157-4cd8468975a4 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:15.391010 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ed84a-080e-44a9-a157-4cd8468975a4 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:15.394172 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:15.433705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e6243-a050-4c8a-89a8-b5eb2c6d9a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:15.434321 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e6243-a050-4c8a-89a8-b5eb2c6d9a0e None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:15.434622 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e6243-a050-4c8a-89a8-b5eb2c6d9a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:15.434894 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e6243-a050-4c8a-89a8-b5eb2c6d9a0e None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:15.435536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e6243-a050-4c8a-89a8-b5eb2c6d9a0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:15.435830 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e6243-a050-4c8a-89a8-b5eb2c6d9a0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:15.437266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e6243-a050-4c8a-89a8-b5eb2c6d9a0e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:15.450910 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 609/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:15.456770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7a7d1-e0e2-4080-a4c6-2e1ba814872d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:15.467534 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:15.582625 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7a7d1-e0e2-4080-a4c6-2e1ba814872d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:15.623942 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e51d36-56fb-42e5-8e89-7a1866c3c8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:15.624611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e51d36-56fb-42e5-8e89-7a1866c3c8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:15.624897 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e51d36-56fb-42e5-8e89-7a1866c3c8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:15.625164 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e51d36-56fb-42e5-8e89-7a1866c3c8d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:15.633700 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e51d36-56fb-42e5-8e89-7a1866c3c8d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:15.636782 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e51d36-56fb-42e5-8e89-7a1866c3c8d6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:15.651360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-90e51d36-56fb-42e5-8e89-7a1866c3c8d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:03:15.651360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:15.651360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:15.651360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:15.651360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:15.651360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:15.651360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:15.651360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:15.651360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:15.651360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:15.651360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:15.651360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:15.651360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:03:15.651360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:03:15.651360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:15.652613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:15.660341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:03:15.660341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:03:15.660341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:03:15.660341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:03:15.660341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:03:15.660341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:03:15.660341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:03:15.660341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:15.660341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 621/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:03:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:15.660925 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e6243-a050-4c8a-89a8-b5eb2c6d9a0e None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:15.787149 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e6243-a050-4c8a-89a8-b5eb2c6d9a0e None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:15.834225 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c3756-4975-41ce-9412-ed88f1b2fbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:15.835003 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c3756-4975-41ce-9412-ed88f1b2fbbc None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:15.835450 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c3756-4975-41ce-9412-ed88f1b2fbbc None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:15.835686 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c3756-4975-41ce-9412-ed88f1b2fbbc None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:15.836444 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c3756-4975-41ce-9412-ed88f1b2fbbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:15.836913 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c3756-4975-41ce-9412-ed88f1b2fbbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:15.839368 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c3756-4975-41ce-9412-ed88f1b2fbbc None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:15.855455 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 610/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:03:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:15.862086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e51d36-56fb-42e5-8e89-7a1866c3c8d6 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:15.949213 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:15.998039 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e51d36-56fb-42e5-8e89-7a1866c3c8d6 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:16.039509 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40a8bb-9a9d-4b71-86c0-8d881b81dfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:16.040221 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40a8bb-9a9d-4b71-86c0-8d881b81dfb9 None admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:16.041216 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40a8bb-9a9d-4b71-86c0-8d881b81dfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:16.041425 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40a8bb-9a9d-4b71-86c0-8d881b81dfb9 None admin] PATH_INFO: `/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/0e16f51a2c714b2fb3d7875b589f9a3c/roles/3209a3d044b84a0ebd7cf1edf2da7120` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:16.061383 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e40a8bb-9a9d-4b71-86c0-8d881b81dfb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0b43dead1664475b9dd52113e08d7ed, user_id=0e16f51a2c714b2fb3d7875b589f9a3c, role_id=3209a3d044b84a0ebd7cf1edf2da7120)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:16.062984 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e40a8bb-9a9d-4b71-86c0-8d881b81dfb9 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:16.077661 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 622/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:03:15 2022] PUT /identity/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/0e16f51a2c714b2fb3d7875b589f9a3c/roles/3209a3d044b84a0ebd7cf1edf2da7120 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 06:03:18.164360 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135025e-c329-4a0e-85d8-ed1c399238a9 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:18.164882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135025e-c329-4a0e-85d8-ed1c399238a9 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:18.164882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135025e-c329-4a0e-85d8-ed1c399238a9 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:18.166469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 611/1233] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:03:18.172551 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ffa66-aba9-4fd2-a576-478189854426 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:18.173024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ffa66-aba9-4fd2-a576-478189854426 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:18.173140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ffa66-aba9-4fd2-a576-478189854426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:18.193954 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-744ffa66-aba9-4fd2-a576-478189854426 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:03:18.195334 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-744ffa66-aba9-4fd2-a576-478189854426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:03:18.280936 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 623/1234] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:03:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:03:18.575530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135025e-c329-4a0e-85d8-ed1c399238a9 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:18.605058 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:18.715741 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135025e-c329-4a0e-85d8-ed1c399238a9 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:18.757262 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734a41d-21ae-4c3a-a723-65a33d94d8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:18.758049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734a41d-21ae-4c3a-a723-65a33d94d8fb None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:18.758341 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734a41d-21ae-4c3a-a723-65a33d94d8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:18.758619 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734a41d-21ae-4c3a-a723-65a33d94d8fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:18.771536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734a41d-21ae-4c3a-a723-65a33d94d8fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-8734a41d-21ae-4c3a-a723-65a33d94d8fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:03:18.777683 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:18.778690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:18.780233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:18.780233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:18.780233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:18.780233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:18.780233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:03:18.780233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:03:18.780233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:03:18.780233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:03:18.780233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:03:18.780233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:03:18.780233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 06:03:18.780233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:18.780233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 612/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Aug 23 06:03:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:03:18.785606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ffa66-aba9-4fd2-a576-478189854426 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:18.836748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:18.896456 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ffa66-aba9-4fd2-a576-478189854426 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:18.936941 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8870b-fd87-4f3e-9ded-efd76bdf6161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:18.937612 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8870b-fd87-4f3e-9ded-efd76bdf6161 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:18.937900 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8870b-fd87-4f3e-9ded-efd76bdf6161 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:18.938168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8870b-fd87-4f3e-9ded-efd76bdf6161 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:18.938896 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8870b-fd87-4f3e-9ded-efd76bdf6161 None admin] RBAC: Adding query filter params (name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:18.939200 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8870b-fd87-4f3e-9ded-efd76bdf6161 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:18.940699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8870b-fd87-4f3e-9ded-efd76bdf6161 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:18.950552 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 624/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Aug 23 06:03:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:03:18.955815 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734a41d-21ae-4c3a-a723-65a33d94d8fb None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:19.054117 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:19.084133 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734a41d-21ae-4c3a-a723-65a33d94d8fb None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:19.128672 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da771ac-0cd2-4217-8b24-835da79abc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:19.129206 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da771ac-0cd2-4217-8b24-835da79abc90 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:19.129480 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da771ac-0cd2-4217-8b24-835da79abc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:19.129746 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da771ac-0cd2-4217-8b24-835da79abc90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:19.142506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da771ac-0cd2-4217-8b24-835da79abc90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:19.144659 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da771ac-0cd2-4217-8b24-835da79abc90 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-0da771ac-0cd2-4217-8b24-835da79abc90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:03:19.168617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:19.170076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:03:19.171189 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:03:19.171189 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:19.171189 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:19.171189 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 613/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:19.175785 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8870b-fd87-4f3e-9ded-efd76bdf6161 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:19.309173 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8870b-fd87-4f3e-9ded-efd76bdf6161 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:19.320985 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:19.358118 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f37ae0-8ca7-435c-8e9b-a73f7146b430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:19.358767 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f37ae0-8ca7-435c-8e9b-a73f7146b430 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:19.359040 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f37ae0-8ca7-435c-8e9b-a73f7146b430 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:19.359356 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f37ae0-8ca7-435c-8e9b-a73f7146b430 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:19.359996 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f37ae0-8ca7-435c-8e9b-a73f7146b430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:19.360295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f37ae0-8ca7-435c-8e9b-a73f7146b430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:19.361842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f37ae0-8ca7-435c-8e9b-a73f7146b430 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:19.377112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 625/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:19.382954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da771ac-0cd2-4217-8b24-835da79abc90 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:19.519129 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da771ac-0cd2-4217-8b24-835da79abc90 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:19.557766 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242bb7c-5ee6-4c03-ad66-aea63abd6c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:19.558695 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242bb7c-5ee6-4c03-ad66-aea63abd6c17 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:19.559102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242bb7c-5ee6-4c03-ad66-aea63abd6c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:19.559482 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242bb7c-5ee6-4c03-ad66-aea63abd6c17 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:19.563436 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:19.569389 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242bb7c-5ee6-4c03-ad66-aea63abd6c17 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:19.571310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242bb7c-5ee6-4c03-ad66-aea63abd6c17 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-6242bb7c-5ee6-4c03-ad66-aea63abd6c17 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:03:19.588638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:19.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:19.591102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:19.591102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:19.591102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:03:19.591102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:03:19.591102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:03:19.591102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:03:19.591102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:03:19.591102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:03:19.591102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 06:03:19.591102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:19.597262 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 614/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:03:19.598464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f37ae0-8ca7-435c-8e9b-a73f7146b430 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:19.721849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f37ae0-8ca7-435c-8e9b-a73f7146b430 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:19.777699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d920b-31d2-4d06-8f2b-e41555ac8ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:19.778980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d920b-31d2-4d06-8f2b-e41555ac8ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:19.779628 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d920b-31d2-4d06-8f2b-e41555ac8ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:19.780161 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d920b-31d2-4d06-8f2b-e41555ac8ea1 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:19.781575 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d920b-31d2-4d06-8f2b-e41555ac8ea1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:19.782120 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d920b-31d2-4d06-8f2b-e41555ac8ea1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:19.784979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d920b-31d2-4d06-8f2b-e41555ac8ea1 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:19.820748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 626/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 23 06:03:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:19.824845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242bb7c-5ee6-4c03-ad66-aea63abd6c17 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:19.964418 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242bb7c-5ee6-4c03-ad66-aea63abd6c17 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:20.003546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b32ea-464c-4b23-a8a1-661ba10fd74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:20.004308 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b32ea-464c-4b23-a8a1-661ba10fd74d None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:20.004588 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b32ea-464c-4b23-a8a1-661ba10fd74d None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:20.004860 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b32ea-464c-4b23-a8a1-661ba10fd74d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:20.081016 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:20.089395 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b32ea-464c-4b23-a8a1-661ba10fd74d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:20.091094 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b32ea-464c-4b23-a8a1-661ba10fd74d None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:20.222141 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 615/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:03:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:03:20.224188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d920b-31d2-4d06-8f2b-e41555ac8ea1 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:20.257926 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:20.368239 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d920b-31d2-4d06-8f2b-e41555ac8ea1 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:20.413760 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee597aa-91f7-4e00-b685-96f14333f5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:20.414430 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee597aa-91f7-4e00-b685-96f14333f5ad None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:20.414768 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee597aa-91f7-4e00-b685-96f14333f5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:20.415017 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee597aa-91f7-4e00-b685-96f14333f5ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:20.427849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee597aa-91f7-4e00-b685-96f14333f5ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:20.429324 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee597aa-91f7-4e00-b685-96f14333f5ad None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-2ee597aa-91f7-4e00-b685-96f14333f5ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:03:20.446433 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:20.447877 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:03:20.453679 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:03:20.453679 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:20.453679 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:20.453679 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 627/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:03:20.455726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b32ea-464c-4b23-a8a1-661ba10fd74d None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:20.566906 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:20.578462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b32ea-464c-4b23-a8a1-661ba10fd74d None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:20.624629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbccd5a-bb32-4cf9-ad07-d10e831eed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:20.626026 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbccd5a-bb32-4cf9-ad07-d10e831eed3a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:20.626746 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbccd5a-bb32-4cf9-ad07-d10e831eed3a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:20.627465 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbccd5a-bb32-4cf9-ad07-d10e831eed3a None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:20.628823 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbccd5a-bb32-4cf9-ad07-d10e831eed3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:20.629568 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbccd5a-bb32-4cf9-ad07-d10e831eed3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:20.632330 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbccd5a-bb32-4cf9-ad07-d10e831eed3a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:20.648839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 616/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:20.655039 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee597aa-91f7-4e00-b685-96f14333f5ad None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:20.761202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:20.794147 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee597aa-91f7-4e00-b685-96f14333f5ad None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:20.834574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690ec12-1beb-423c-ba27-2a63ae99d395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:20.835146 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690ec12-1beb-423c-ba27-2a63ae99d395 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:20.835449 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690ec12-1beb-423c-ba27-2a63ae99d395 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:20.835727 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690ec12-1beb-423c-ba27-2a63ae99d395 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:20.843999 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690ec12-1beb-423c-ba27-2a63ae99d395 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:20.845791 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690ec12-1beb-423c-ba27-2a63ae99d395 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:20.861845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-8690ec12-1beb-423c-ba27-2a63ae99d395 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:03:20.861845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:20.861845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:20.861845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:20.861845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:20.861845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:20.861845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:20.861845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:20.861845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:20.861845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:20.861845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:20.861845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:20.861845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:03:20.861845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:03:20.861845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:20.869232 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:20.870464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:03:20.870464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:03:20.870464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:03:20.870464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:03:20.870464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:03:20.870464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:03:20.870464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:03:20.870464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:20.870464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 628/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:03:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:20.871801 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbccd5a-bb32-4cf9-ad07-d10e831eed3a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:20.999733 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbccd5a-bb32-4cf9-ad07-d10e831eed3a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:21.042299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da2c46-42a5-4b8f-bef9-fcfc2f95d1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:21.042299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da2c46-42a5-4b8f-bef9-fcfc2f95d1ff None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:21.042783 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da2c46-42a5-4b8f-bef9-fcfc2f95d1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:21.042874 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da2c46-42a5-4b8f-bef9-fcfc2f95d1ff None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:21.043568 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da2c46-42a5-4b8f-bef9-fcfc2f95d1ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:21.043868 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da2c46-42a5-4b8f-bef9-fcfc2f95d1ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:21.045340 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da2c46-42a5-4b8f-bef9-fcfc2f95d1ff None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:21.059586 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 617/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:03:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:21.065806 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690ec12-1beb-423c-ba27-2a63ae99d395 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:21.201979 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690ec12-1beb-423c-ba27-2a63ae99d395 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:21.247993 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702fcaa-12bf-49cd-abbe-b1349df1101a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:21.248642 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702fcaa-12bf-49cd-abbe-b1349df1101a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:21.248912 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702fcaa-12bf-49cd-abbe-b1349df1101a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:21.249249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702fcaa-12bf-49cd-abbe-b1349df1101a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:21.250159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702fcaa-12bf-49cd-abbe-b1349df1101a None admin] RBAC: Adding query filter params (role.id=3209a3d044b84a0ebd7cf1edf2da7120, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=0e16f51a2c714b2fb3d7875b589f9a3c) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:21.250572 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702fcaa-12bf-49cd-abbe-b1349df1101a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:21.252010 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702fcaa-12bf-49cd-abbe-b1349df1101a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:21.261048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 629/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:03:21 2022] GET /identity/v3/role_assignments?user.id=0e16f51a2c714b2fb3d7875b589f9a3c&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=3209a3d044b84a0ebd7cf1edf2da7120 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 06:03:23.640521 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfad574-6207-41c8-9bd1-885f38b92448 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:23.642387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfad574-6207-41c8-9bd1-885f38b92448 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:23.642387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfad574-6207-41c8-9bd1-885f38b92448 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:23.642869 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 618/1247] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:03:23.649246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035557ba-5926-4114-9018-35347f5b3602 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:23.650446 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035557ba-5926-4114-9018-35347f5b3602 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:23.650643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035557ba-5926-4114-9018-35347f5b3602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:23.658445 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:23.673668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-035557ba-5926-4114-9018-35347f5b3602 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:03:23.675228 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-035557ba-5926-4114-9018-35347f5b3602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:03:23.769279 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 630/1248] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:03:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:03:24.081998 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfad574-6207-41c8-9bd1-885f38b92448 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:24.104994 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:24.214533 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfad574-6207-41c8-9bd1-885f38b92448 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:24.256139 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326562a2-7476-406b-ab4c-fe3621c3db5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:24.256808 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326562a2-7476-406b-ab4c-fe3621c3db5b None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:24.257093 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326562a2-7476-406b-ab4c-fe3621c3db5b None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:24.257405 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326562a2-7476-406b-ab4c-fe3621c3db5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:24.270484 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326562a2-7476-406b-ab4c-fe3621c3db5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-326562a2-7476-406b-ab4c-fe3621c3db5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:03:24.276111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:24.277077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:24.278543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:24.278543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:24.278543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:24.278543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:24.278543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:03:24.278543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:03:24.278543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:03:24.278543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:03:24.278543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:03:24.278543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:03:24.278543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:03:24.278543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:24.278543 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 619/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:03:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:24.285657 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035557ba-5926-4114-9018-35347f5b3602 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:24.414169 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035557ba-5926-4114-9018-35347f5b3602 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:24.448083 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:24.454731 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe6806-92cc-4857-be11-d60c52792741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:24.455389 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe6806-92cc-4857-be11-d60c52792741 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:24.455668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe6806-92cc-4857-be11-d60c52792741 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:24.455977 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe6806-92cc-4857-be11-d60c52792741 None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:24.456714 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe6806-92cc-4857-be11-d60c52792741 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:24.457024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe6806-92cc-4857-be11-d60c52792741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:24.458568 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe6806-92cc-4857-be11-d60c52792741 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:24.469167 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 631/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:03:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:24.474983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326562a2-7476-406b-ab4c-fe3621c3db5b None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:24.576926 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:24.604333 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326562a2-7476-406b-ab4c-fe3621c3db5b None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:24.645134 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dce385-03f9-4f0c-9043-a641955d7dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:24.645852 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dce385-03f9-4f0c-9043-a641955d7dfc None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:24.646082 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dce385-03f9-4f0c-9043-a641955d7dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:24.646316 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dce385-03f9-4f0c-9043-a641955d7dfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:24.658297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dce385-03f9-4f0c-9043-a641955d7dfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:24.659818 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dce385-03f9-4f0c-9043-a641955d7dfc None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-b5dce385-03f9-4f0c-9043-a641955d7dfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:03:24.675036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:24.676323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:03:24.677375 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:03:24.677375 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:24.677375 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:24.677375 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 620/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:24.683480 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe6806-92cc-4857-be11-d60c52792741 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:24.829721 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe6806-92cc-4857-be11-d60c52792741 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:24.879561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ae717-0f96-4b76-916b-59c393257200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:24.880398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ae717-0f96-4b76-916b-59c393257200 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:24.880753 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ae717-0f96-4b76-916b-59c393257200 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:24.881234 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ae717-0f96-4b76-916b-59c393257200 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:24.882062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ae717-0f96-4b76-916b-59c393257200 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:24.882457 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ae717-0f96-4b76-916b-59c393257200 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:24.883887 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ae717-0f96-4b76-916b-59c393257200 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:24.906856 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 632/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:24.913198 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dce385-03f9-4f0c-9043-a641955d7dfc None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:25.051934 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dce385-03f9-4f0c-9043-a641955d7dfc None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:25.082417 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:25.091338 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded130d2-5372-44ec-9090-d459c112cd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:25.092308 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded130d2-5372-44ec-9090-d459c112cd59 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:25.093045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded130d2-5372-44ec-9090-d459c112cd59 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:25.093045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded130d2-5372-44ec-9090-d459c112cd59 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:25.102331 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded130d2-5372-44ec-9090-d459c112cd59 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:25.104640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded130d2-5372-44ec-9090-d459c112cd59 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-ded130d2-5372-44ec-9090-d459c112cd59 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:03:25.120037 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:25.121076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:25.122936 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:25.122936 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:25.122936 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:03:25.122936 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:03:25.122936 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:03:25.122936 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:03:25.122936 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:03:25.122936 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:03:25.122936 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 06:03:25.122936 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:25.122936 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 621/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:25.128496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ae717-0f96-4b76-916b-59c393257200 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:25.198078 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:25.253406 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ae717-0f96-4b76-916b-59c393257200 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:25.305649 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c213428-fbe3-4b2e-91de-06e27c4808bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:25.307068 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c213428-fbe3-4b2e-91de-06e27c4808bd None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:25.307639 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c213428-fbe3-4b2e-91de-06e27c4808bd None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:25.307948 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c213428-fbe3-4b2e-91de-06e27c4808bd None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:25.308822 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c213428-fbe3-4b2e-91de-06e27c4808bd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:25.309122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c213428-fbe3-4b2e-91de-06e27c4808bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:25.312010 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c213428-fbe3-4b2e-91de-06e27c4808bd None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:25.341860 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 633/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 23 06:03:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:25.348643 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded130d2-5372-44ec-9090-d459c112cd59 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:25.472329 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded130d2-5372-44ec-9090-d459c112cd59 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:25.518604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9103d3a-62a4-4009-8613-4d11cba793f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:25.519280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9103d3a-62a4-4009-8613-4d11cba793f7 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:25.519557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9103d3a-62a4-4009-8613-4d11cba793f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:25.519831 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9103d3a-62a4-4009-8613-4d11cba793f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:25.593260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:25.610390 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9103d3a-62a4-4009-8613-4d11cba793f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:25.611914 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9103d3a-62a4-4009-8613-4d11cba793f7 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:25.735451 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 622/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:03:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:03:25.741548 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c213428-fbe3-4b2e-91de-06e27c4808bd None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:25.869132 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c213428-fbe3-4b2e-91de-06e27c4808bd None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:25.885430 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:25.912220 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02482a3f-2fff-4efc-820f-1eaec2316e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:25.912861 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02482a3f-2fff-4efc-820f-1eaec2316e3b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:25.913189 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02482a3f-2fff-4efc-820f-1eaec2316e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:25.913411 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02482a3f-2fff-4efc-820f-1eaec2316e3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:25.926206 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02482a3f-2fff-4efc-820f-1eaec2316e3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:25.927944 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02482a3f-2fff-4efc-820f-1eaec2316e3b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-02482a3f-2fff-4efc-820f-1eaec2316e3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:03:25.945730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:25.952270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:03:25.953251 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:03:25.953251 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:25.953251 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:25.953251 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 634/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:25.953493 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9103d3a-62a4-4009-8613-4d11cba793f7 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:26.057347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:26.070587 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9103d3a-62a4-4009-8613-4d11cba793f7 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:26.111963 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2e5d0-4d03-4054-9a35-b5841853e245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:26.112657 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2e5d0-4d03-4054-9a35-b5841853e245 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:26.112961 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2e5d0-4d03-4054-9a35-b5841853e245 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:26.113229 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2e5d0-4d03-4054-9a35-b5841853e245 None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:26.113975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2e5d0-4d03-4054-9a35-b5841853e245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:26.114295 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2e5d0-4d03-4054-9a35-b5841853e245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:26.115776 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2e5d0-4d03-4054-9a35-b5841853e245 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:26.132639 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 623/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:26.138870 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02482a3f-2fff-4efc-820f-1eaec2316e3b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:26.298915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02482a3f-2fff-4efc-820f-1eaec2316e3b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:26.339793 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef6136-5fdd-4100-bd79-eac169698bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:26.340635 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef6136-5fdd-4100-bd79-eac169698bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:26.340993 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef6136-5fdd-4100-bd79-eac169698bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:26.341338 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef6136-5fdd-4100-bd79-eac169698bd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:26.351886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef6136-5fdd-4100-bd79-eac169698bd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:26.354483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef6136-5fdd-4100-bd79-eac169698bd7 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:26.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-72ef6136-5fdd-4100-bd79-eac169698bd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:03:26.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:26.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:26.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:26.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:26.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:26.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:26.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:26.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:26.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:26.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:26.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:26.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:03:26.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:03:26.370442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:26.372307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:26.373358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:03:26.373358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:03:26.373358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:03:26.373358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:03:26.373358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:03:26.373358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:03:26.373358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:03:26.373358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:26.373358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 635/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:03:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:26.378744 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2e5d0-4d03-4054-9a35-b5841853e245 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:26.531837 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2e5d0-4d03-4054-9a35-b5841853e245 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:26.583955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c50d1-d5c3-4e49-bc82-b736d66e6e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:26.584581 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c50d1-d5c3-4e49-bc82-b736d66e6e39 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:26.584859 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c50d1-d5c3-4e49-bc82-b736d66e6e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:26.585132 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c50d1-d5c3-4e49-bc82-b736d66e6e39 None admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:26.585839 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c50d1-d5c3-4e49-bc82-b736d66e6e39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:26.586202 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c50d1-d5c3-4e49-bc82-b736d66e6e39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:26.588772 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c50d1-d5c3-4e49-bc82-b736d66e6e39 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:26.603739 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 624/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:03:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:26.610584 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef6136-5fdd-4100-bd79-eac169698bd7 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:26.641240 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:26.752522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef6136-5fdd-4100-bd79-eac169698bd7 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:26.804084 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b89e1-bf91-43a8-9eba-668d0073f669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:26.805277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b89e1-bf91-43a8-9eba-668d0073f669 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:26.805277 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b89e1-bf91-43a8-9eba-668d0073f669 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:26.805494 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b89e1-bf91-43a8-9eba-668d0073f669 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:26.806504 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b89e1-bf91-43a8-9eba-668d0073f669 None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=0e16f51a2c714b2fb3d7875b589f9a3c) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:26.807355 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b89e1-bf91-43a8-9eba-668d0073f669 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:26.808825 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b89e1-bf91-43a8-9eba-668d0073f669 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:26.822894 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 636/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:03:26 2022] GET /identity/v3/role_assignments?user.id=0e16f51a2c714b2fb3d7875b589f9a3c&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:28.884799 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3da8a-f806-4e89-8aca-35ec365f1351 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:28.885369 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3da8a-f806-4e89-8aca-35ec365f1351 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:28.885682 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3da8a-f806-4e89-8aca-35ec365f1351 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:28.887144 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 625/1261] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:03:28.893790 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b341caf-20b2-4d1f-b3a6-779a766a73d6 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:28.894442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b341caf-20b2-4d1f-b3a6-779a766a73d6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:28.894442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b341caf-20b2-4d1f-b3a6-779a766a73d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:28.914666 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4b341caf-20b2-4d1f-b3a6-779a766a73d6 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:03:28.916561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4b341caf-20b2-4d1f-b3a6-779a766a73d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:03:29.000908 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 637/1262] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:03:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:03:29.281052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3da8a-f806-4e89-8aca-35ec365f1351 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:29.295099 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:29.396766 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3da8a-f806-4e89-8aca-35ec365f1351 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:29.432833 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560adee4-a409-42f1-88c4-aac9b1d0316c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:29.433463 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560adee4-a409-42f1-88c4-aac9b1d0316c None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:29.433742 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560adee4-a409-42f1-88c4-aac9b1d0316c None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:29.434014 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560adee4-a409-42f1-88c4-aac9b1d0316c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:29.444398 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560adee4-a409-42f1-88c4-aac9b1d0316c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-560adee4-a409-42f1-88c4-aac9b1d0316c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:03:29.450347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:29.451506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:29.452382 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:29.452382 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:29.452382 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:29.452382 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:29.452382 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:03:29.452382 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:03:29.452382 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:03:29.452382 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:03:29.452382 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:03:29.452382 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:03:29.452382 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:03:29.452382 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:29.452382 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 626/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:03:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:29.457378 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b341caf-20b2-4d1f-b3a6-779a766a73d6 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:29.477622 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:29.569392 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b341caf-20b2-4d1f-b3a6-779a766a73d6 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:29.607137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f975e5f-4838-4839-a724-410128262f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:29.607737 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f975e5f-4838-4839-a724-410128262f8a None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:29.608011 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f975e5f-4838-4839-a724-410128262f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:29.608279 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f975e5f-4838-4839-a724-410128262f8a None admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:29.608935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f975e5f-4838-4839-a724-410128262f8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:29.609233 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f975e5f-4838-4839-a724-410128262f8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:29.610616 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f975e5f-4838-4839-a724-410128262f8a None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:29.618719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 638/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:03:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:29.624829 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560adee4-a409-42f1-88c4-aac9b1d0316c None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:29.693761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:29.731023 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560adee4-a409-42f1-88c4-aac9b1d0316c None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:29.768432 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc26401-0fac-4a50-89d1-0884d21817ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:29.769058 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc26401-0fac-4a50-89d1-0884d21817ec None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:29.769335 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc26401-0fac-4a50-89d1-0884d21817ec None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:29.769604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc26401-0fac-4a50-89d1-0884d21817ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:29.780428 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc26401-0fac-4a50-89d1-0884d21817ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:29.781879 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc26401-0fac-4a50-89d1-0884d21817ec None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-4cc26401-0fac-4a50-89d1-0884d21817ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:03:29.797059 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:29.798072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:03:29.798996 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:03:29.798996 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:29.798996 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:29.798996 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 627/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:29.804129 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f975e5f-4838-4839-a724-410128262f8a None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:29.890486 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:29.911889 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f975e5f-4838-4839-a724-410128262f8a None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:29.948086 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ef70b-9542-4455-8da1-d7cf5f901dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:29.949014 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ef70b-9542-4455-8da1-d7cf5f901dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:29.949290 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ef70b-9542-4455-8da1-d7cf5f901dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:29.949565 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ef70b-9542-4455-8da1-d7cf5f901dd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:29.950210 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ef70b-9542-4455-8da1-d7cf5f901dd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:29.950525 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ef70b-9542-4455-8da1-d7cf5f901dd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:29.951934 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ef70b-9542-4455-8da1-d7cf5f901dd5 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:29.965363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 639/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:29.971163 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc26401-0fac-4a50-89d1-0884d21817ec None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:30.070665 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc26401-0fac-4a50-89d1-0884d21817ec None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:30.093570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:30.109688 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e45a84-b32f-4453-afca-b64eaa70599f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:30.110321 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e45a84-b32f-4453-afca-b64eaa70599f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:30.110629 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e45a84-b32f-4453-afca-b64eaa70599f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:30.110904 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e45a84-b32f-4453-afca-b64eaa70599f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:30.121256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e45a84-b32f-4453-afca-b64eaa70599f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:30.123013 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e45a84-b32f-4453-afca-b64eaa70599f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-e2e45a84-b32f-4453-afca-b64eaa70599f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:03:30.135434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:30.136462 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:30.137387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:30.137387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:30.137387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:03:30.137387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:03:30.137387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:03:30.137387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:03:30.137387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:03:30.137387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:03:30.137387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 06:03:30.137387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:30.137387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 628/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:30.142458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ef70b-9542-4455-8da1-d7cf5f901dd5 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:30.241949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ef70b-9542-4455-8da1-d7cf5f901dd5 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:30.277014 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a15cd-3a98-49d3-b132-7b91ad860ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:30.277664 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a15cd-3a98-49d3-b132-7b91ad860ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:30.277992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a15cd-3a98-49d3-b132-7b91ad860ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:30.278266 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a15cd-3a98-49d3-b132-7b91ad860ae2 None admin] PATH_INFO: `/v3/users` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:30.279192 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a15cd-3a98-49d3-b132-7b91ad860ae2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:30.279487 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a15cd-3a98-49d3-b132-7b91ad860ae2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:30.281077 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a15cd-3a98-49d3-b132-7b91ad860ae2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:30.292679 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:30.304288 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 640/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 23 06:03:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:30.311186 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e45a84-b32f-4453-afca-b64eaa70599f None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:30.417849 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e45a84-b32f-4453-afca-b64eaa70599f None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:30.452267 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c81c0-435b-4dce-bdb7-c1bc29ac32f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:30.452888 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c81c0-435b-4dce-bdb7-c1bc29ac32f2 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:30.453165 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c81c0-435b-4dce-bdb7-c1bc29ac32f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:30.453434 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c81c0-435b-4dce-bdb7-c1bc29ac32f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:30.464875 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c81c0-435b-4dce-bdb7-c1bc29ac32f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:30.466314 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c81c0-435b-4dce-bdb7-c1bc29ac32f2 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-7c5c81c0-435b-4dce-bdb7-c1bc29ac32f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:03:30.480828 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:30.481965 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:03:30.482942 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:03:30.482942 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:30.482942 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:30.482942 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 629/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:30.487825 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a15cd-3a98-49d3-b132-7b91ad860ae2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:30.591870 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a15cd-3a98-49d3-b132-7b91ad860ae2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:30.626897 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822bc32-bffb-4769-a096-315ba0435cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:30.627507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822bc32-bffb-4769-a096-315ba0435cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:30.627788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822bc32-bffb-4769-a096-315ba0435cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:30.628061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822bc32-bffb-4769-a096-315ba0435cf5 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:30.628703 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822bc32-bffb-4769-a096-315ba0435cf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:30.629006 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822bc32-bffb-4769-a096-315ba0435cf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:30.630580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822bc32-bffb-4769-a096-315ba0435cf5 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:30.642716 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 641/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:30.648456 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c81c0-435b-4dce-bdb7-c1bc29ac32f2 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:30.685119 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:30.756653 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c81c0-435b-4dce-bdb7-c1bc29ac32f2 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:30.791630 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87bc2e-5625-43cc-a1bc-c5555e800beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:30.792258 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87bc2e-5625-43cc-a1bc-c5555e800beb None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:30.792537 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87bc2e-5625-43cc-a1bc-c5555e800beb None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:30.792802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87bc2e-5625-43cc-a1bc-c5555e800beb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:30.799751 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87bc2e-5625-43cc-a1bc-c5555e800beb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:30.801193 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87bc2e-5625-43cc-a1bc-c5555e800beb None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:30.812460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-cb87bc2e-5625-43cc-a1bc-c5555e800beb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:03:30.812460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:30.812460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:30.812460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:30.812460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:30.812460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:30.812460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:30.812460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:30.812460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:30.812460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:30.812460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:30.812460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:30.812460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:03:30.812460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:03:30.812460 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:30.813534 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:30.819539 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:03:30.819539 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:03:30.819539 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:03:30.819539 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:03:30.819539 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:03:30.819539 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:03:30.819539 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:03:30.819539 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:30.819539 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 630/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:03:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:30.820133 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822bc32-bffb-4769-a096-315ba0435cf5 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:30.879836 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:30.923995 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822bc32-bffb-4769-a096-315ba0435cf5 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:30.960843 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dafa4-27ff-4726-98bd-ab6642945a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:30.961459 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dafa4-27ff-4726-98bd-ab6642945a86 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:30.961739 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dafa4-27ff-4726-98bd-ab6642945a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:30.962011 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dafa4-27ff-4726-98bd-ab6642945a86 None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:30.964046 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9dafa4-27ff-4726-98bd-ab6642945a86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:30.964356 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9dafa4-27ff-4726-98bd-ab6642945a86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:30.965826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9dafa4-27ff-4726-98bd-ab6642945a86 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:30.978613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 642/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:03:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:30.984637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87bc2e-5625-43cc-a1bc-c5555e800beb None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:31.091033 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:31.100501 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87bc2e-5625-43cc-a1bc-c5555e800beb None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:31.134977 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01077af5-306b-4b93-a5c3-7fa67d7d13c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:31.135719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01077af5-306b-4b93-a5c3-7fa67d7d13c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:31.135994 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01077af5-306b-4b93-a5c3-7fa67d7d13c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:31.136260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01077af5-306b-4b93-a5c3-7fa67d7d13c1 None admin] PATH_INFO: `/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/0e16f51a2c714b2fb3d7875b589f9a3c/roles/f6acb6fdca5a44de9ba00b2b0d4d3202` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:31.138249 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01077af5-306b-4b93-a5c3-7fa67d7d13c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0b43dead1664475b9dd52113e08d7ed, user_id=0e16f51a2c714b2fb3d7875b589f9a3c, role_id=f6acb6fdca5a44de9ba00b2b0d4d3202)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:31.139736 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01077af5-306b-4b93-a5c3-7fa67d7d13c1 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:31.154312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 631/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Aug 23 06:03:30 2022] PUT /identity/v3/projects/b0b43dead1664475b9dd52113e08d7ed/users/0e16f51a2c714b2fb3d7875b589f9a3c/roles/f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:03:33.126083 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d5767-0ce2-475e-9e63-1ba2de84bd9b None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:33.126742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d5767-0ce2-475e-9e63-1ba2de84bd9b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:33.126742 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d5767-0ce2-475e-9e63-1ba2de84bd9b None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:33.128056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 643/1274] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:03:33.133459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafd1a7-19bb-491d-a337-63de05b2bb4a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:33.133893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafd1a7-19bb-491d-a337-63de05b2bb4a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:33.134607 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafd1a7-19bb-491d-a337-63de05b2bb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:33.153475 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-0aafd1a7-19bb-491d-a337-63de05b2bb4a None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:03:33.154770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0aafd1a7-19bb-491d-a337-63de05b2bb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:03:33.223255 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 632/1275] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:03:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:03:33.526248 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d5767-0ce2-475e-9e63-1ba2de84bd9b None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:33.631435 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d5767-0ce2-475e-9e63-1ba2de84bd9b None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:33.649838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:33.667344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9b3a4-2b6e-4551-89f9-53408d2380eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:33.667968 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9b3a4-2b6e-4551-89f9-53408d2380eb None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:33.668241 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9b3a4-2b6e-4551-89f9-53408d2380eb None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:33.668507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9b3a4-2b6e-4551-89f9-53408d2380eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:33.678559 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b9b3a4-2b6e-4551-89f9-53408d2380eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-d3b9b3a4-2b6e-4551-89f9-53408d2380eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 06:03:33.683634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:33.684788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:33.685987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:33.685987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:33.685987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:33.685987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:33.685987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 06:03:33.685987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 06:03:33.685987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 06:03:33.685987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 06:03:33.685987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 06:03:33.685987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 06:03:33.685987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 06:03:33.685987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:33.685987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 644/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Aug 23 06:03:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:33.690699 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafd1a7-19bb-491d-a337-63de05b2bb4a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:33.778377 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:33.797685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafd1a7-19bb-491d-a337-63de05b2bb4a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:33.834194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215b438-354f-481d-8a79-51ac032318cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:33.834836 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215b438-354f-481d-8a79-51ac032318cd None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:33.835115 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215b438-354f-481d-8a79-51ac032318cd None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:33.835384 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215b438-354f-481d-8a79-51ac032318cd None admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:33.836041 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215b438-354f-481d-8a79-51ac032318cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:33.836340 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215b438-354f-481d-8a79-51ac032318cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:33.837820 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215b438-354f-481d-8a79-51ac032318cd None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:33.846096 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 633/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Aug 23 06:03:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:33.851581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9b3a4-2b6e-4551-89f9-53408d2380eb None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:33.958803 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9b3a4-2b6e-4551-89f9-53408d2380eb None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:33.992855 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e0ac7-7306-4528-a454-83fef184899d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:33.994067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e0ac7-7306-4528-a454-83fef184899d None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:33.994344 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e0ac7-7306-4528-a454-83fef184899d None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:33.994647 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e0ac7-7306-4528-a454-83fef184899d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:34.005656 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e0ac7-7306-4528-a454-83fef184899d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:34.007131 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e0ac7-7306-4528-a454-83fef184899d None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-457e0ac7-7306-4528-a454-83fef184899d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:03:34.021718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:34.022920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:03:34.023836 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:03:34.023836 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:34.023836 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:34.023836 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 645/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:34.030124 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215b438-354f-481d-8a79-51ac032318cd None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:34.133497 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215b438-354f-481d-8a79-51ac032318cd None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:34.167919 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c784972-bfb0-4de5-ae66-36133f96708a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:34.168527 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c784972-bfb0-4de5-ae66-36133f96708a None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:34.168802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c784972-bfb0-4de5-ae66-36133f96708a None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:34.169068 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c784972-bfb0-4de5-ae66-36133f96708a None admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:34.169706 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c784972-bfb0-4de5-ae66-36133f96708a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:34.169999 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c784972-bfb0-4de5-ae66-36133f96708a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:34.171397 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c784972-bfb0-4de5-ae66-36133f96708a None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:34.183685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 634/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:34.189452 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e0ac7-7306-4528-a454-83fef184899d None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:34.222485 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:34.298685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e0ac7-7306-4528-a454-83fef184899d None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:34.334295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa166873-6fe0-4e4d-aff3-7c93030e54c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:34.334957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa166873-6fe0-4e4d-aff3-7c93030e54c5 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:34.335273 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa166873-6fe0-4e4d-aff3-7c93030e54c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:34.335509 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa166873-6fe0-4e4d-aff3-7c93030e54c5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:34.343528 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa166873-6fe0-4e4d-aff3-7c93030e54c5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:34.344961 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa166873-6fe0-4e4d-aff3-7c93030e54c5 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-aa166873-6fe0-4e4d-aff3-7c93030e54c5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 06:03:34.356070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:34.357088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:34.358029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:34.358029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:34.358029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 06:03:34.358029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 06:03:34.358029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 06:03:34.358029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 06:03:34.358029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 06:03:34.358029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 06:03:34.358029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 06:03:34.358029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:34.358029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 646/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:34.362986 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c784972-bfb0-4de5-ae66-36133f96708a None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:34.366807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:34.465111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c784972-bfb0-4de5-ae66-36133f96708a None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:34.499246 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08829c83-349b-48cc-bd81-f4898654bb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:34.499848 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08829c83-349b-48cc-bd81-f4898654bb74 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:34.500123 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08829c83-349b-48cc-bd81-f4898654bb74 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:34.500389 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08829c83-349b-48cc-bd81-f4898654bb74 None admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:34.501110 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08829c83-349b-48cc-bd81-f4898654bb74 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:34.501408 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08829c83-349b-48cc-bd81-f4898654bb74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:34.502862 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08829c83-349b-48cc-bd81-f4898654bb74 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:34.520301 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 635/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Aug 23 06:03:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:34.526202 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa166873-6fe0-4e4d-aff3-7c93030e54c5 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:34.620965 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:34.631270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa166873-6fe0-4e4d-aff3-7c93030e54c5 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:34.665741 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4550870-f0ea-4e63-b56a-1d96fb2f5ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:34.666747 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4550870-f0ea-4e63-b56a-1d96fb2f5ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:34.667029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4550870-f0ea-4e63-b56a-1d96fb2f5ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:34.667307 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4550870-f0ea-4e63-b56a-1d96fb2f5ab5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:34.738732 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4550870-f0ea-4e63-b56a-1d96fb2f5ab5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:34.739820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4550870-f0ea-4e63-b56a-1d96fb2f5ab5 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:34.835529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:34.846557 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 647/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Aug 23 06:03:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 06:03:34.852282 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08829c83-349b-48cc-bd81-f4898654bb74 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:34.916300 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:34.961397 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08829c83-349b-48cc-bd81-f4898654bb74 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:34.997646 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56226c5a-96f9-4452-8e0c-504bb0ce8052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:34.998269 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56226c5a-96f9-4452-8e0c-504bb0ce8052 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:34.998588 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56226c5a-96f9-4452-8e0c-504bb0ce8052 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:34.998859 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56226c5a-96f9-4452-8e0c-504bb0ce8052 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:35.008824 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56226c5a-96f9-4452-8e0c-504bb0ce8052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:35.010870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56226c5a-96f9-4452-8e0c-504bb0ce8052 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-56226c5a-96f9-4452-8e0c-504bb0ce8052 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 06:03:35.025208 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:35.026237 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 06:03:35.027183 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 06:03:35.027183 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 06:03:35.027183 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:35.027183 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 636/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:35.031988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4550870-f0ea-4e63-b56a-1d96fb2f5ab5 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:35.135570 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4550870-f0ea-4e63-b56a-1d96fb2f5ab5 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:35.169980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932343f-b42c-4521-9612-9c32b97023e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:35.170529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932343f-b42c-4521-9612-9c32b97023e2 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:35.170883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932343f-b42c-4521-9612-9c32b97023e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:35.171154 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932343f-b42c-4521-9612-9c32b97023e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:35.171788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932343f-b42c-4521-9612-9c32b97023e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:35.172088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932343f-b42c-4521-9612-9c32b97023e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:35.173451 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932343f-b42c-4521-9612-9c32b97023e2 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:35.190224 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 648/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Aug 23 06:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:35.192127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56226c5a-96f9-4452-8e0c-504bb0ce8052 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:35.309414 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56226c5a-96f9-4452-8e0c-504bb0ce8052 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:35.324340 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:35.346305 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4134d31-5d65-46e7-815b-1a213d2a5b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:35.346964 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4134d31-5d65-46e7-815b-1a213d2a5b28 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:35.347245 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4134d31-5d65-46e7-815b-1a213d2a5b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:35.347516 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4134d31-5d65-46e7-815b-1a213d2a5b28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:35.355026 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4134d31-5d65-46e7-815b-1a213d2a5b28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:35.356465 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4134d31-5d65-46e7-815b-1a213d2a5b28 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:35.368561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application [None req-c4134d31-5d65-46e7-815b-1a213d2a5b28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:03:35.368561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:35.368561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:35.368561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:35.368561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:35.368561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:35.368561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:35.368561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:35.368561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:35.368561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:35.368561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:35.368561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:35.368561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 06:03:35.368561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 06:03:35.368561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:35.369488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:35.370378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 06:03:35.370378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 06:03:35.370378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 06:03:35.370378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 06:03:35.370378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 06:03:35.370378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 06:03:35.370378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 06:03:35.370378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: ERROR keystone.server.flask.application Aug 23 06:03:35.370378 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 637/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Aug 23 06:03:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:35.375849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932343f-b42c-4521-9612-9c32b97023e2 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:35.428606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:35.481082 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932343f-b42c-4521-9612-9c32b97023e2 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:35.519788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608b31b-a959-453c-af39-d282fcde6a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:35.520398 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608b31b-a959-453c-af39-d282fcde6a6c None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:35.520679 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608b31b-a959-453c-af39-d282fcde6a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:35.520953 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608b31b-a959-453c-af39-d282fcde6a6c None admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:35.521643 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608b31b-a959-453c-af39-d282fcde6a6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:35.521950 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608b31b-a959-453c-af39-d282fcde6a6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:35.523407 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608b31b-a959-453c-af39-d282fcde6a6c None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:35.536910 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 649/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Aug 23 06:03:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:35.543018 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4134d31-5d65-46e7-815b-1a213d2a5b28 None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:35.641763 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4134d31-5d65-46e7-815b-1a213d2a5b28 None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:35.676477 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8efbb1-30bc-4c1d-bbb3-84962f18f88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:35.677069 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8efbb1-30bc-4c1d-bbb3-84962f18f88f None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:35.677343 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8efbb1-30bc-4c1d-bbb3-84962f18f88f None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:35.677608 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8efbb1-30bc-4c1d-bbb3-84962f18f88f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:35.678328 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8efbb1-30bc-4c1d-bbb3-84962f18f88f None admin] RBAC: Adding query filter params (role.id=f6acb6fdca5a44de9ba00b2b0d4d3202, scope.project.id=b0b43dead1664475b9dd52113e08d7ed, user.id=0e16f51a2c714b2fb3d7875b589f9a3c) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:35.678706 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8efbb1-30bc-4c1d-bbb3-84962f18f88f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:35.680070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8efbb1-30bc-4c1d-bbb3-84962f18f88f None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:35.686526 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 638/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Aug 23 06:03:35 2022] GET /identity/v3/role_assignments?user.id=0e16f51a2c714b2fb3d7875b589f9a3c&scope.project.id=b0b43dead1664475b9dd52113e08d7ed&role.id=f6acb6fdca5a44de9ba00b2b0d4d3202 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:37.667452 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095b144-68db-4c27-8180-a5dfb75a3166 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:37.667987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095b144-68db-4c27-8180-a5dfb75a3166 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:37.667987 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095b144-68db-4c27-8180-a5dfb75a3166 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:37.669319 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 650/1288] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:03:37.675049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2ea90-99c3-483f-8796-77f70d7bfb93 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:37.675589 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2ea90-99c3-483f-8796-77f70d7bfb93 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:37.675692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2ea90-99c3-483f-8796-77f70d7bfb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:37.694977 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-16b2ea90-99c3-483f-8796-77f70d7bfb93 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:03:37.696167 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-16b2ea90-99c3-483f-8796-77f70d7bfb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:03:37.728054 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:37.766635 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 639/1289] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:03:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:03:38.040291 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095b144-68db-4c27-8180-a5dfb75a3166 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:38.151462 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095b144-68db-4c27-8180-a5dfb75a3166 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:38.162574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:38.186118 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d8d19-eb7e-43c3-ba20-884a5c9a94cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:38.186765 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d8d19-eb7e-43c3-ba20-884a5c9a94cf None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:38.187037 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d8d19-eb7e-43c3-ba20-884a5c9a94cf None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:38.187304 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d8d19-eb7e-43c3-ba20-884a5c9a94cf None admin] PATH_INFO: `/v3/services/placement` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:38.187873 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d8d19-eb7e-43c3-ba20-884a5c9a94cf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:38.189223 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d8d19-eb7e-43c3-ba20-884a5c9a94cf None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-942d8d19-eb7e-43c3-ba20-884a5c9a94cf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:38.198159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:38.199227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:38.200135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:38.200135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:38.200135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:38.200135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:38.200135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:38.200135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:38.200135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:38.200135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:38.200135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:38.200135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:38.200135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:03:38.200135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:03:38.200135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 06:03:38.200135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:38.200135 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 651/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 23 06:03:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:38.205430 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2ea90-99c3-483f-8796-77f70d7bfb93 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:38.311624 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2ea90-99c3-483f-8796-77f70d7bfb93 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:38.356699 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a3b88-f72c-4cbe-9f38-ba06ebc16cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:38.357390 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a3b88-f72c-4cbe-9f38-ba06ebc16cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:38.357689 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a3b88-f72c-4cbe-9f38-ba06ebc16cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:38.357992 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a3b88-f72c-4cbe-9f38-ba06ebc16cb4 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:38.358761 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a3b88-f72c-4cbe-9f38-ba06ebc16cb4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:38.359153 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a3b88-f72c-4cbe-9f38-ba06ebc16cb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:38.360623 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a3b88-f72c-4cbe-9f38-ba06ebc16cb4 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:38.366437 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 640/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Aug 23 06:03:38 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:38.373948 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d8d19-eb7e-43c3-ba20-884a5c9a94cf None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:38.487777 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d8d19-eb7e-43c3-ba20-884a5c9a94cf None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:38.523223 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732e1c8-7e33-4c4d-9976-1dea63dde1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:38.523835 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732e1c8-7e33-4c4d-9976-1dea63dde1fa None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:38.524115 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732e1c8-7e33-4c4d-9976-1dea63dde1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:38.524387 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732e1c8-7e33-4c4d-9976-1dea63dde1fa None admin] PATH_INFO: `/v3/services` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:38.525377 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732e1c8-7e33-4c4d-9976-1dea63dde1fa None admin] RBAC: Adding query filter params (type=placement) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:38.525690 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732e1c8-7e33-4c4d-9976-1dea63dde1fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:38.527111 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732e1c8-7e33-4c4d-9976-1dea63dde1fa None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:38.531710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 652/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Aug 23 06:03:38 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:40.455812 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96af2e-7ffe-457b-ae48-5a658e694a4e None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:40.456348 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96af2e-7ffe-457b-ae48-5a658e694a4e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:40.456348 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96af2e-7ffe-457b-ae48-5a658e694a4e None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:40.457695 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 641/1293] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:03:40.463926 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5c244-a602-477a-820b-15435e545165 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:40.464371 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5c244-a602-477a-820b-15435e545165 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:40.464462 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5c244-a602-477a-820b-15435e545165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:40.484529 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-34f5c244-a602-477a-820b-15435e545165 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:03:40.485713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-34f5c244-a602-477a-820b-15435e545165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:03:40.517772 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:40.557257 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 653/1294] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:03:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:03:40.830622 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96af2e-7ffe-457b-ae48-5a658e694a4e None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:40.863700 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:40.941687 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96af2e-7ffe-457b-ae48-5a658e694a4e None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:40.978172 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84e2fe-05d5-469e-a86c-4d901deb729e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:40.978816 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84e2fe-05d5-469e-a86c-4d901deb729e None admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:40.979099 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84e2fe-05d5-469e-a86c-4d901deb729e None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:40.979370 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84e2fe-05d5-469e-a86c-4d901deb729e None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:40.980032 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84e2fe-05d5-469e-a86c-4d901deb729e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:40.981409 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84e2fe-05d5-469e-a86c-4d901deb729e None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:40.997819 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 642/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Aug 23 06:03:40 2022] POST /identity/v3/services => generated 246 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:03:42.909036 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73368f3-0edd-4fc1-9274-a886eca14940 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:42.909582 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73368f3-0edd-4fc1-9274-a886eca14940 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:42.909582 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73368f3-0edd-4fc1-9274-a886eca14940 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:42.911048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 654/1296] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:03:42.916871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a4c1b-882c-46d9-8161-a826beeb58e2 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:42.917327 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a4c1b-882c-46d9-8161-a826beeb58e2 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:42.917435 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a4c1b-882c-46d9-8161-a826beeb58e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:42.936296 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-af3a4c1b-882c-46d9-8161-a826beeb58e2 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:03:42.937479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-af3a4c1b-882c-46d9-8161-a826beeb58e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:03:43.020172 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 643/1297] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:03:42 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 06:03:43.313014 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73368f3-0edd-4fc1-9274-a886eca14940 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:43.429845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73368f3-0edd-4fc1-9274-a886eca14940 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:43.465019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0795d-b966-481b-85a9-119559358b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:43.465647 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0795d-b966-481b-85a9-119559358b9b None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:43.465920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0795d-b966-481b-85a9-119559358b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:43.466182 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0795d-b966-481b-85a9-119559358b9b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:43.466823 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b0795d-b966-481b-85a9-119559358b9b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:43.468178 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b0795d-b966-481b-85a9-119559358b9b None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-73b0795d-b966-481b-85a9-119559358b9b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:43.476775 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:43.477838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:43.478935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:43.478935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:43.478935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:43.478935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:43.478935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:43.478935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:43.478935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:43.478935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:43.478935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:43.478935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:43.478935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:03:43.478935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:03:43.478935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 06:03:43.478935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:43.478935 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 655/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 23 06:03:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 06:03:43.484106 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a4c1b-882c-46d9-8161-a826beeb58e2 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:43.504070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:43.590697 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a4c1b-882c-46d9-8161-a826beeb58e2 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:43.627529 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dde719-3f36-47d4-a28c-6937746d71ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:43.628141 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dde719-3f36-47d4-a28c-6937746d71ef None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:43.628422 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dde719-3f36-47d4-a28c-6937746d71ef None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:43.628701 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dde719-3f36-47d4-a28c-6937746d71ef None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:43.629330 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dde719-3f36-47d4-a28c-6937746d71ef None admin] RBAC: Adding query filter params (name=placement) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:43.629632 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dde719-3f36-47d4-a28c-6937746d71ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:43.631039 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dde719-3f36-47d4-a28c-6937746d71ef None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:43.636763 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 644/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Aug 23 06:03:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:43.642896 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0795d-b966-481b-85a9-119559358b9b None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:43.675634 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:43.746668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0795d-b966-481b-85a9-119559358b9b None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:43.781095 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36359d-aab3-4ffb-8955-952c70d51ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:43.781708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36359d-aab3-4ffb-8955-952c70d51ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:43.781989 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36359d-aab3-4ffb-8955-952c70d51ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:43.782258 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36359d-aab3-4ffb-8955-952c70d51ae6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:43.783051 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36359d-aab3-4ffb-8955-952c70d51ae6 None admin] RBAC: Adding query filter params (interface=public, service_id=0e4ae9a4b9394115a1473c06376bf1f5, region_id=RegionOne) {{(pid=81747) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:43.783358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36359d-aab3-4ffb-8955-952c70d51ae6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:43.784717 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36359d-aab3-4ffb-8955-952c70d51ae6 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:43.789840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 656/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Aug 23 06:03:43 2022] GET /identity/v3/endpoints?service_id=0e4ae9a4b9394115a1473c06376bf1f5&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:43.797023 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dde719-3f36-47d4-a28c-6937746d71ef None admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:43.884325 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:43.907781 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dde719-3f36-47d4-a28c-6937746d71ef None admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:43.952017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a829787-eb44-475f-87c6-9877a89f8a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:43.952017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a829787-eb44-475f-87c6-9877a89f8a45 None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:43.952017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a829787-eb44-475f-87c6-9877a89f8a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:43.952017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a829787-eb44-475f-87c6-9877a89f8a45 None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:43.952827 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a829787-eb44-475f-87c6-9877a89f8a45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:43.953472 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a829787-eb44-475f-87c6-9877a89f8a45 None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:43.960845 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 645/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Aug 23 06:03:43 2022] GET /identity/v3/services => generated 2191 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 06:03:45.911043 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cd888-3269-4903-b103-68e21bf43c2c None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:45.911589 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cd888-3269-4903-b103-68e21bf43c2c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:45.911589 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cd888-3269-4903-b103-68e21bf43c2c None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:45.912918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 657/1302] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:03:45.919017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94afb12-f20c-4515-934d-4a392ff3ca48 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:45.919496 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94afb12-f20c-4515-934d-4a392ff3ca48 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:45.919597 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94afb12-f20c-4515-934d-4a392ff3ca48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:45.937580 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-d94afb12-f20c-4515-934d-4a392ff3ca48 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:03:45.938770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d94afb12-f20c-4515-934d-4a392ff3ca48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:03:46.012317 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 646/1303] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:03:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 06:03:46.293368 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cd888-3269-4903-b103-68e21bf43c2c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:46.406328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:46.414286 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cd888-3269-4903-b103-68e21bf43c2c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:46.453474 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871e80e-e509-4f18-9a10-042e2fc63d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:46.454106 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871e80e-e509-4f18-9a10-042e2fc63d54 None admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:46.454387 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871e80e-e509-4f18-9a10-042e2fc63d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:46.454700 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871e80e-e509-4f18-9a10-042e2fc63d54 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:46.455292 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871e80e-e509-4f18-9a10-042e2fc63d54 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:46.456644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871e80e-e509-4f18-9a10-042e2fc63d54 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application [None req-8871e80e-e509-4f18-9a10-042e2fc63d54 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 06:03:46.465957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 06:03:46.467072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 06:03:46.468125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 06:03:46.468125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application with Lock( Aug 23 06:03:46.468125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 06:03:46.468125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self._enter() Aug 23 06:03:46.468125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 06:03:46.468125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 06:03:46.468125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 06:03:46.468125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application return self.creator() Aug 23 06:03:46.468125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 06:03:46.468125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application created_value = creator( Aug 23 06:03:46.468125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 06:03:46.468125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 06:03:46.468125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 06:03:46.468125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: ERROR keystone.server.flask.application Aug 23 06:03:46.468125 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 658/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Aug 23 06:03:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 06:03:46.474727 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94afb12-f20c-4515-934d-4a392ff3ca48 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:46.555644 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:46.579353 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94afb12-f20c-4515-934d-4a392ff3ca48 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:46.613201 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599bfb1-2369-4ca9-a9d8-f88402773eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:46.613799 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599bfb1-2369-4ca9-a9d8-f88402773eda None admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:46.614074 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599bfb1-2369-4ca9-a9d8-f88402773eda None admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:46.614344 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599bfb1-2369-4ca9-a9d8-f88402773eda None admin] PATH_INFO: `/v3/services` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:46.615030 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599bfb1-2369-4ca9-a9d8-f88402773eda None admin] RBAC: Adding query filter params (name=placement) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:03:46.615324 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599bfb1-2369-4ca9-a9d8-f88402773eda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:46.616679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599bfb1-2369-4ca9-a9d8-f88402773eda None admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:46.626610 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 647/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Aug 23 06:03:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:03:46.628062 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871e80e-e509-4f18-9a10-042e2fc63d54 None admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:03:46.732247 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871e80e-e509-4f18-9a10-042e2fc63d54 None admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:03:46.768431 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae816392-6ce1-4684-b71b-49dbef336709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:03:46.769044 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae816392-6ce1-4684-b71b-49dbef336709 None admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:03:46.769324 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae816392-6ce1-4684-b71b-49dbef336709 None admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:03:46.769596 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae816392-6ce1-4684-b71b-49dbef336709 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:03:46.770259 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae816392-6ce1-4684-b71b-49dbef336709 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:03:46.771664 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae816392-6ce1-4684-b71b-49dbef336709 None admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:03:46.797592 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 659/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 23 06:03:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:03:56.619932 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=30 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:03:56.791099 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:04:56.346024 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8a18e-51ce-4cda-a6b0-f8244c22b1f7 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:04:56.346639 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8a18e-51ce-4cda-a6b0-f8244c22b1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:04:56.346987 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8a18e-51ce-4cda-a6b0-f8244c22b1f7 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:04:56.354023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01318c88-7739-49ec-8bd6-1248019a7845 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:04:56.354606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01318c88-7739-49ec-8bd6-1248019a7845 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:04:56.354606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01318c88-7739-49ec-8bd6-1248019a7845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:04:56.359075 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 648/1307] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:04:56 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:04:56.483326 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-01318c88-7739-49ec-8bd6-1248019a7845 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:04:56.484549 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-01318c88-7739-49ec-8bd6-1248019a7845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:04:56.744547 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 660/1308] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:04:57.076874 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561588bd-6574-47cd-b15e-33d7c2c15e57 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:04:57.077160 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561588bd-6574-47cd-b15e-33d7c2c15e57 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:04:57.077431 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561588bd-6574-47cd-b15e-33d7c2c15e57 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:04:57.079242 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 649/1309] 10.222.0.20 () {54 vars in 965 bytes} [Tue Aug 23 06:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:04:57.086341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a56b3-96a5-4998-b045-15de6115682e None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:04:57.087168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a56b3-96a5-4998-b045-15de6115682e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:04:57.087441 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a56b3-96a5-4998-b045-15de6115682e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:04:57.286504 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ab3a56b3-96a5-4998-b045-15de6115682e None None] MFA Rules not processed for user `dae9eff3bbf04496af2300a24c630eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:04:57.287926 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ab3a56b3-96a5-4998-b045-15de6115682e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:04:57.336017 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:04:57.458034 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 661/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Tue Aug 23 06:04:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 06:04:57.468094 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561588bd-6574-47cd-b15e-33d7c2c15e57 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:04:57.469871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-561588bd-6574-47cd-b15e-33d7c2c15e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:04:57.764995 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561588bd-6574-47cd-b15e-33d7c2c15e57 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:04:57.889365 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b459a89-f41c-40e7-a869-3541cbd97294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:04:57.890324 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b459a89-f41c-40e7-a869-3541cbd97294 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:04:57.890734 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b459a89-f41c-40e7-a869-3541cbd97294 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:04:57.891111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b459a89-f41c-40e7-a869-3541cbd97294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:04:57.892866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4b459a89-f41c-40e7-a869-3541cbd97294 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:04:57.956764 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:04:58.020432 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b459a89-f41c-40e7-a869-3541cbd97294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:04:58.022130 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b459a89-f41c-40e7-a869-3541cbd97294 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:04:58.044064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 650/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:05:00.566959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f85927-4585-4fd2-b817-07a1c68e2abd None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:00.568243 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f85927-4585-4fd2-b817-07a1c68e2abd None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:00.568738 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f85927-4585-4fd2-b817-07a1c68e2abd None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:00.576943 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f484b-c9cb-4a7b-93f3-d8cfdaa306de None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:00.577907 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f484b-c9cb-4a7b-93f3-d8cfdaa306de None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:00.578417 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f484b-c9cb-4a7b-93f3-d8cfdaa306de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:00.582287 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 662/1312] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:05:00 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:00.658733 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-dc6f484b-c9cb-4a7b-93f3-d8cfdaa306de None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:00.661127 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-dc6f484b-c9cb-4a7b-93f3-d8cfdaa306de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:00.772489 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 651/1313] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 06:05:01.076613 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f85927-4585-4fd2-b817-07a1c68e2abd None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:05:01.078309 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b0f85927-4585-4fd2-b817-07a1c68e2abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:01.212345 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f85927-4585-4fd2-b817-07a1c68e2abd None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:05:01.295778 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:01.298758 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935f186-68ed-4160-bf41-746c7be12b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:05:01.299409 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935f186-68ed-4160-bf41-746c7be12b55 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:01.299844 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935f186-68ed-4160-bf41-746c7be12b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:01.299844 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935f186-68ed-4160-bf41-746c7be12b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:01.302124 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3935f186-68ed-4160-bf41-746c7be12b55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:01.426025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935f186-68ed-4160-bf41-746c7be12b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:05:01.427473 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935f186-68ed-4160-bf41-746c7be12b55 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:05:01.446679 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 663/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:05:03.722854 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807742d-650e-4d1c-926b-80fc488882e7 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:03.723358 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807742d-650e-4d1c-926b-80fc488882e7 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:03.723358 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807742d-650e-4d1c-926b-80fc488882e7 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:03.724723 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 652/1315] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:03.730453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7cd8d-fb1b-49b6-a7ee-af548888758a None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:03.730957 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7cd8d-fb1b-49b6-a7ee-af548888758a None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:03.731146 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7cd8d-fb1b-49b6-a7ee-af548888758a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:03.798725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-2dc7cd8d-fb1b-49b6-a7ee-af548888758a None None] MFA Rules not processed for user `20bcd5c59c07488b83a3391be92d86f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:03.799916 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2dc7cd8d-fb1b-49b6-a7ee-af548888758a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:03.853227 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:03.900308 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 664/1316] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:05:03 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:04.189985 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807742d-650e-4d1c-926b-80fc488882e7 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:05:04.191604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2807742d-650e-4d1c-926b-80fc488882e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:04.202900 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:04.307326 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807742d-650e-4d1c-926b-80fc488882e7 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:05:04.382726 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce20f4-ca41-4fb5-98ab-e2ceb55c54c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:05:04.383461 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce20f4-ca41-4fb5-98ab-e2ceb55c54c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:04.383850 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce20f4-ca41-4fb5-98ab-e2ceb55c54c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:04.384247 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce20f4-ca41-4fb5-98ab-e2ceb55c54c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:04.385870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-26ce20f4-ca41-4fb5-98ab-e2ceb55c54c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:04.452486 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:04.502984 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce20f4-ca41-4fb5-98ab-e2ceb55c54c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:05:04.504433 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce20f4-ca41-4fb5-98ab-e2ceb55c54c3 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:05:04.523143 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 653/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:05:04 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:05:07.070854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749c01a-cade-4b96-9d0d-6c3b40aaefcb None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:07.071464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749c01a-cade-4b96-9d0d-6c3b40aaefcb None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:07.071464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749c01a-cade-4b96-9d0d-6c3b40aaefcb None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:07.072745 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 665/1318] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:07.078234 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46392fc8-abb6-44c1-9b1f-6f0956b82d36 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:07.078664 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46392fc8-abb6-44c1-9b1f-6f0956b82d36 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:07.078804 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46392fc8-abb6-44c1-9b1f-6f0956b82d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:07.151145 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-46392fc8-abb6-44c1-9b1f-6f0956b82d36 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:07.152367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-46392fc8-abb6-44c1-9b1f-6f0956b82d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:07.249693 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 654/1319] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:05:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:07.538269 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749c01a-cade-4b96-9d0d-6c3b40aaefcb None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:05:07.540567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0749c01a-cade-4b96-9d0d-6c3b40aaefcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:07.644931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749c01a-cade-4b96-9d0d-6c3b40aaefcb None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:05:07.722496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494ec66-183d-4b3d-9257-a6c6d0b2cf56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:05:07.723149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494ec66-183d-4b3d-9257-a6c6d0b2cf56 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:07.723426 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494ec66-183d-4b3d-9257-a6c6d0b2cf56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:07.723701 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494ec66-183d-4b3d-9257-a6c6d0b2cf56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:07.725437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d494ec66-183d-4b3d-9257-a6c6d0b2cf56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:07.736988 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:07.847848 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d494ec66-183d-4b3d-9257-a6c6d0b2cf56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:05:07.849216 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d494ec66-183d-4b3d-9257-a6c6d0b2cf56 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:05:07.866990 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 666/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:05:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:05:10.668566 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd48864-aea8-405c-8457-cae6f1b45daa None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:10.669075 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd48864-aea8-405c-8457-cae6f1b45daa None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:10.669075 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd48864-aea8-405c-8457-cae6f1b45daa None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:10.670544 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 655/1321] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:10.676032 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e5637-cb3c-465d-83a9-4b0e3d69b4ba None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:10.676862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e5637-cb3c-465d-83a9-4b0e3d69b4ba None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:10.677425 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e5637-cb3c-465d-83a9-4b0e3d69b4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:10.753611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-e70e5637-cb3c-465d-83a9-4b0e3d69b4ba None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:10.754846 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e70e5637-cb3c-465d-83a9-4b0e3d69b4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:10.815473 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:10.866280 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 667/1322] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:11.175424 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0a6eb-78cf-48df-aeb4-7e2e1b21840f None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:11.175702 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0a6eb-78cf-48df-aeb4-7e2e1b21840f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:11.175973 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0a6eb-78cf-48df-aeb4-7e2e1b21840f None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:11.177366 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 656/1323] 10.222.0.20 () {54 vars in 965 bytes} [Tue Aug 23 06:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:11.185747 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abb603-65c7-4915-a688-aebb9308efa8 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:11.186197 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abb603-65c7-4915-a688-aebb9308efa8 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:11.186604 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abb603-65c7-4915-a688-aebb9308efa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:11.273418 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-a8abb603-65c7-4915-a688-aebb9308efa8 None None] MFA Rules not processed for user `dae9eff3bbf04496af2300a24c630eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:11.275016 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a8abb603-65c7-4915-a688-aebb9308efa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:11.357691 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 668/1324] 10.222.0.20 () {58 vars in 1062 bytes} [Tue Aug 23 06:05:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:11.366770 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0a6eb-78cf-48df-aeb4-7e2e1b21840f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:05:11.368955 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-34a0a6eb-78cf-48df-aeb4-7e2e1b21840f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:11.399177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:11.505420 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0a6eb-78cf-48df-aeb4-7e2e1b21840f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:05:11.627204 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c03ea-db35-4e72-924b-d8c1b6798297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:05:11.628561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c03ea-db35-4e72-924b-d8c1b6798297 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:11.628843 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c03ea-db35-4e72-924b-d8c1b6798297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:11.629115 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c03ea-db35-4e72-924b-d8c1b6798297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:11.630765 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-761c03ea-db35-4e72-924b-d8c1b6798297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:11.711097 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:11.753812 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c03ea-db35-4e72-924b-d8c1b6798297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:05:11.755312 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c03ea-db35-4e72-924b-d8c1b6798297 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:05:11.777029 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 657/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:05:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:05:15.095065 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303172f-3775-4a87-8d13-e053f5b9d055 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:15.095510 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303172f-3775-4a87-8d13-e053f5b9d055 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:15.095510 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303172f-3775-4a87-8d13-e053f5b9d055 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:15.098796 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 669/1326] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:05:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:15.102257 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31613e13-bbc0-4ee6-a524-73d8119ca1c3 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:15.102786 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31613e13-bbc0-4ee6-a524-73d8119ca1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:15.102786 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31613e13-bbc0-4ee6-a524-73d8119ca1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:15.174012 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-31613e13-bbc0-4ee6-a524-73d8119ca1c3 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:15.178433 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-31613e13-bbc0-4ee6-a524-73d8119ca1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:15.290699 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 658/1327] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:15.585543 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303172f-3775-4a87-8d13-e053f5b9d055 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:05:15.587271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6303172f-3775-4a87-8d13-e053f5b9d055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:15.662696 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:15.711027 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303172f-3775-4a87-8d13-e053f5b9d055 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:05:15.796601 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b26f28-9619-43bf-be48-5ce8069020a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:05:15.797206 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b26f28-9619-43bf-be48-5ce8069020a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:15.797491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b26f28-9619-43bf-be48-5ce8069020a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:15.797758 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b26f28-9619-43bf-be48-5ce8069020a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:15.799412 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b5b26f28-9619-43bf-be48-5ce8069020a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:15.967586 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:15.967586 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b26f28-9619-43bf-be48-5ce8069020a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:05:15.967586 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b26f28-9619-43bf-be48-5ce8069020a1 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:05:15.974223 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 670/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:05:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:05:18.300118 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decd26a-53c6-4ca8-9ebc-55b01054c377 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:18.301036 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decd26a-53c6-4ca8-9ebc-55b01054c377 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:18.301565 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decd26a-53c6-4ca8-9ebc-55b01054c377 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:18.304321 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 659/1329] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:05:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:18.311470 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca2ca7-022a-4c0b-88b1-c5bf70f104b4 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:18.312001 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca2ca7-022a-4c0b-88b1-c5bf70f104b4 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:18.312001 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca2ca7-022a-4c0b-88b1-c5bf70f104b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:18.405061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-4bca2ca7-022a-4c0b-88b1-c5bf70f104b4 None None] MFA Rules not processed for user `20bcd5c59c07488b83a3391be92d86f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:18.406662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-4bca2ca7-022a-4c0b-88b1-c5bf70f104b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:18.517816 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 671/1330] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:05:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:18.804457 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decd26a-53c6-4ca8-9ebc-55b01054c377 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:05:18.806041 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7decd26a-53c6-4ca8-9ebc-55b01054c377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:18.843983 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:18.935236 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decd26a-53c6-4ca8-9ebc-55b01054c377 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:05:19.033865 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917ac52-666a-4c32-9f24-753013535229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:05:19.034473 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917ac52-666a-4c32-9f24-753013535229 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:19.034791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917ac52-666a-4c32-9f24-753013535229 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:19.035055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917ac52-666a-4c32-9f24-753013535229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:19.036661 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7917ac52-666a-4c32-9f24-753013535229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:19.125561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:19.166077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917ac52-666a-4c32-9f24-753013535229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:05:19.167377 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917ac52-666a-4c32-9f24-753013535229 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:05:19.187072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 660/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:05:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:05:21.525168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34614a-3af7-496e-9ca4-7e9ef0f79a75 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:21.525704 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34614a-3af7-496e-9ca4-7e9ef0f79a75 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:21.525922 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34614a-3af7-496e-9ca4-7e9ef0f79a75 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:21.528245 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 672/1332] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:05:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:21.534357 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd96ab-cc6e-4d6f-8231-e191d5729357 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:21.535298 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd96ab-cc6e-4d6f-8231-e191d5729357 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:21.535298 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd96ab-cc6e-4d6f-8231-e191d5729357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:21.610307 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-7abd96ab-cc6e-4d6f-8231-e191d5729357 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:21.611565 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7abd96ab-cc6e-4d6f-8231-e191d5729357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:21.722440 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 661/1333] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:22.013950 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34614a-3af7-496e-9ca4-7e9ef0f79a75 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:05:22.015515 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ec34614a-3af7-496e-9ca4-7e9ef0f79a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:22.086205 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:22.123222 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34614a-3af7-496e-9ca4-7e9ef0f79a75 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:05:22.207493 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c8ced-86a2-416d-a73d-419b43c17a15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:05:22.208106 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c8ced-86a2-416d-a73d-419b43c17a15 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:22.208393 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c8ced-86a2-416d-a73d-419b43c17a15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:22.208664 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c8ced-86a2-416d-a73d-419b43c17a15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:22.210265 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e12c8ced-86a2-416d-a73d-419b43c17a15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:22.331751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c8ced-86a2-416d-a73d-419b43c17a15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:05:22.333261 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c8ced-86a2-416d-a73d-419b43c17a15 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:05:22.343568 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:22.353043 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 673/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:05:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:05:24.345240 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404076e-4e9e-4559-946a-5189d5f2eb21 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:24.345821 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404076e-4e9e-4559-946a-5189d5f2eb21 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:24.345821 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404076e-4e9e-4559-946a-5189d5f2eb21 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:24.347351 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 662/1335] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:24.352997 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032b349-3f59-418d-bae2-e68ababc86d0 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:24.353442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032b349-3f59-418d-bae2-e68ababc86d0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:24.353661 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032b349-3f59-418d-bae2-e68ababc86d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:24.421768 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-8032b349-3f59-418d-bae2-e68ababc86d0 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:24.422994 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8032b349-3f59-418d-bae2-e68ababc86d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:24.522874 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 674/1336] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:24.812169 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404076e-4e9e-4559-946a-5189d5f2eb21 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:05:24.813703 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6404076e-4e9e-4559-946a-5189d5f2eb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:24.856047 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:24.929495 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404076e-4e9e-4559-946a-5189d5f2eb21 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:05:25.010163 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5175ec-9cc0-4ce8-a930-b95f4f8533b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:05:25.010798 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5175ec-9cc0-4ce8-a930-b95f4f8533b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:25.011075 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5175ec-9cc0-4ce8-a930-b95f4f8533b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:25.011343 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5175ec-9cc0-4ce8-a930-b95f4f8533b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:25.012927 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1e5175ec-9cc0-4ce8-a930-b95f4f8533b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:25.102924 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:25.129958 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5175ec-9cc0-4ce8-a930-b95f4f8533b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:05:25.131441 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5175ec-9cc0-4ce8-a930-b95f4f8533b2 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:05:25.151172 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 663/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:05:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:05:27.848041 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b1bee-cd50-49ad-b979-8908dcf1ae8f None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:27.848597 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b1bee-cd50-49ad-b979-8908dcf1ae8f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:27.848597 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b1bee-cd50-49ad-b979-8908dcf1ae8f None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:27.849925 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 675/1338] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:27.855296 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b42cb-30eb-4497-b599-470ac048a8d8 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:27.855769 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b42cb-30eb-4497-b599-470ac048a8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:27.855885 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b42cb-30eb-4497-b599-470ac048a8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:27.925518 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e52b42cb-30eb-4497-b599-470ac048a8d8 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:27.926767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e52b42cb-30eb-4497-b599-470ac048a8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:28.028087 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 664/1339] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:05:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:28.315807 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b1bee-cd50-49ad-b979-8908dcf1ae8f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:05:28.317607 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-497b1bee-cd50-49ad-b979-8908dcf1ae8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:28.403914 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:28.432581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b1bee-cd50-49ad-b979-8908dcf1ae8f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:05:28.510160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a07e34-1ba4-475a-9a0b-254529919e98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:05:28.510793 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a07e34-1ba4-475a-9a0b-254529919e98 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:28.511068 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a07e34-1ba4-475a-9a0b-254529919e98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:28.511333 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a07e34-1ba4-475a-9a0b-254529919e98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:28.512905 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-37a07e34-1ba4-475a-9a0b-254529919e98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:28.631708 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a07e34-1ba4-475a-9a0b-254529919e98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:05:28.633071 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a07e34-1ba4-475a-9a0b-254529919e98 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:05:28.654188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 676/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:05:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:05:32.751788 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928bc5f-481e-4df2-a157-0773e4bd8595 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:32.752359 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928bc5f-481e-4df2-a157-0773e4bd8595 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:32.752359 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928bc5f-481e-4df2-a157-0773e4bd8595 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:32.753808 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 665/1341] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:32.759289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3f7d2-57d4-476c-9fdf-ad9ecbda5402 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:32.759774 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3f7d2-57d4-476c-9fdf-ad9ecbda5402 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:32.759908 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3f7d2-57d4-476c-9fdf-ad9ecbda5402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:32.765334 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:32.838252 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-b7b3f7d2-57d4-476c-9fdf-ad9ecbda5402 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:32.839123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b7b3f7d2-57d4-476c-9fdf-ad9ecbda5402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:32.957668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 677/1342] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:33.249427 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928bc5f-481e-4df2-a157-0773e4bd8595 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:05:33.251327 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6928bc5f-481e-4df2-a157-0773e4bd8595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:33.318058 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:33.381051 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928bc5f-481e-4df2-a157-0773e4bd8595 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:05:33.463118 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1586d-1889-4af4-afb2-4daf29dff575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:05:33.463834 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1586d-1889-4af4-afb2-4daf29dff575 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:33.464169 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1586d-1889-4af4-afb2-4daf29dff575 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:33.464506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1586d-1889-4af4-afb2-4daf29dff575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:33.466165 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-95b1586d-1889-4af4-afb2-4daf29dff575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:33.568896 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:33.588374 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1586d-1889-4af4-afb2-4daf29dff575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:05:33.589954 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1586d-1889-4af4-afb2-4daf29dff575 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:05:33.609103 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 666/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:05:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:05:36.208297 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa721100-dd73-4a25-9cb0-a65d63a7d68f None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:36.208802 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa721100-dd73-4a25-9cb0-a65d63a7d68f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:36.208802 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa721100-dd73-4a25-9cb0-a65d63a7d68f None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:36.210255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 678/1344] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:05:36.215617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430b057-63b9-423d-a26e-e83be17abd9b None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:36.216190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430b057-63b9-423d-a26e-e83be17abd9b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:36.216190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430b057-63b9-423d-a26e-e83be17abd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:36.290252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-8430b057-63b9-423d-a26e-e83be17abd9b None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:36.291756 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-8430b057-63b9-423d-a26e-e83be17abd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:36.407529 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 667/1345] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:05:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:36.697981 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa721100-dd73-4a25-9cb0-a65d63a7d68f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:05:36.699724 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-aa721100-dd73-4a25-9cb0-a65d63a7d68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:36.796566 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:36.815932 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa721100-dd73-4a25-9cb0-a65d63a7d68f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:05:36.925820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2cb8b-1e86-4e58-9cc3-02dadcea2002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:05:36.926811 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2cb8b-1e86-4e58-9cc3-02dadcea2002 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:36.927242 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2cb8b-1e86-4e58-9cc3-02dadcea2002 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:36.927654 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2cb8b-1e86-4e58-9cc3-02dadcea2002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:36.930835 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-24a2cb8b-1e86-4e58-9cc3-02dadcea2002 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:37.065270 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2cb8b-1e86-4e58-9cc3-02dadcea2002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:05:37.066674 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2cb8b-1e86-4e58-9cc3-02dadcea2002 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:05:37.086221 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 679/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:05:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:05:41.687394 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd34cc0-4f9d-499a-9821-5c6e0e79b579 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:41.687922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd34cc0-4f9d-499a-9821-5c6e0e79b579 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:41.694815 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd34cc0-4f9d-499a-9821-5c6e0e79b579 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:41.696989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 668/1347] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:05:41 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:41.703028 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761231c2-ea55-4789-bcdb-b558acecbad6 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:41.703482 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761231c2-ea55-4789-bcdb-b558acecbad6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:41.703561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761231c2-ea55-4789-bcdb-b558acecbad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:41.720292 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:41.785697 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-761231c2-ea55-4789-bcdb-b558acecbad6 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:41.787205 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-761231c2-ea55-4789-bcdb-b558acecbad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:41.905119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 680/1348] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:42.204204 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd34cc0-4f9d-499a-9821-5c6e0e79b579 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:05:42.205802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4dd34cc0-4f9d-499a-9821-5c6e0e79b579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:42.287380 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:42.339310 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd34cc0-4f9d-499a-9821-5c6e0e79b579 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:05:42.430688 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88f349-80f7-4a47-b0b7-d46c6749b065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:05:42.431224 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88f349-80f7-4a47-b0b7-d46c6749b065 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:42.431486 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88f349-80f7-4a47-b0b7-d46c6749b065 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:42.431757 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88f349-80f7-4a47-b0b7-d46c6749b065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:42.433449 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ad88f349-80f7-4a47-b0b7-d46c6749b065 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:42.556373 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:42.561194 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88f349-80f7-4a47-b0b7-d46c6749b065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:05:42.562916 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88f349-80f7-4a47-b0b7-d46c6749b065 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:05:42.582348 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 669/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:05:48.473520 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42159063-3ba9-48f9-bc1f-5f72beca68c9 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:48.474056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42159063-3ba9-48f9-bc1f-5f72beca68c9 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:48.474056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42159063-3ba9-48f9-bc1f-5f72beca68c9 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:48.476304 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 681/1350] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 23 06:05:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:48.488020 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722b69b-54d0-472e-9267-88b00726d279 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:48.488518 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722b69b-54d0-472e-9267-88b00726d279 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:48.488718 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722b69b-54d0-472e-9267-88b00726d279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:48.635679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-0722b69b-54d0-472e-9267-88b00726d279 None None] MFA Rules not processed for user `0e16f51a2c714b2fb3d7875b589f9a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:48.636902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0722b69b-54d0-472e-9267-88b00726d279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:48.799454 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 670/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 23 06:05:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:51.901905 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=59 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:54.057122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf5e34-6b0b-42ef-983e-8957962a8ecf None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:54.057122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf5e34-6b0b-42ef-983e-8957962a8ecf None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:54.057862 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf5e34-6b0b-42ef-983e-8957962a8ecf None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:54.059003 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 682/1352] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 23 06:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:54.068204 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21699d1-72f3-4fc6-882c-be72bdc3d37f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:54.068751 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21699d1-72f3-4fc6-882c-be72bdc3d37f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:54.069191 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21699d1-72f3-4fc6-882c-be72bdc3d37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:54.142611 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:54.179750 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c21699d1-72f3-4fc6-882c-be72bdc3d37f None None] MFA Rules not processed for user `0e16f51a2c714b2fb3d7875b589f9a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:54.181704 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c21699d1-72f3-4fc6-882c-be72bdc3d37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:54.324750 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 671/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 23 06:05:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:55.970885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fabc9-3757-441f-8f57-fb21de2e9724 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:55.971825 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fabc9-3757-441f-8f57-fb21de2e9724 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:55.972696 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fabc9-3757-441f-8f57-fb21de2e9724 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:55.974471 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 683/1354] 10.222.0.20 () {54 vars in 921 bytes} [Tue Aug 23 06:05:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:55.980256 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195f3b4-bab2-42b4-b769-d682b41bd024 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:55.980713 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195f3b4-bab2-42b4-b769-d682b41bd024 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:55.980945 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195f3b4-bab2-42b4-b769-d682b41bd024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:56.066950 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c195f3b4-bab2-42b4-b769-d682b41bd024 None None] MFA Rules not processed for user `0e16f51a2c714b2fb3d7875b589f9a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:56.086307 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c195f3b4-bab2-42b4-b769-d682b41bd024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:56.132561 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:05:56.211058 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 672/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Aug 23 06:05:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:59.332811 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e7d1f-73f2-4c52-904b-ed914cb3ed91 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:59.333347 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e7d1f-73f2-4c52-904b-ed914cb3ed91 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:59.333347 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e7d1f-73f2-4c52-904b-ed914cb3ed91 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:59.334631 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 684/1356] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:59.340537 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654598c6-afdf-4f31-a4c9-ac28c8f6c6b8 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:59.341081 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654598c6-afdf-4f31-a4c9-ac28c8f6c6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:59.341185 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654598c6-afdf-4f31-a4c9-ac28c8f6c6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:59.416071 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-654598c6-afdf-4f31-a4c9-ac28c8f6c6b8 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:05:59.417287 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-654598c6-afdf-4f31-a4c9-ac28c8f6c6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:05:59.527086 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 673/1357] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:05:59.864434 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458e031-7222-4f6a-9563-7805fc54b41c None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:59.864995 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458e031-7222-4f6a-9563-7805fc54b41c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:59.865423 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458e031-7222-4f6a-9563-7805fc54b41c None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:59.867364 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 685/1358] 10.222.0.20 () {54 vars in 962 bytes} [Tue Aug 23 06:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:05:59.899089 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c0e09-cf0c-4425-aafa-b4b0471ae784 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:05:59.899650 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c0e09-cf0c-4425-aafa-b4b0471ae784 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:05:59.900163 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c0e09-cf0c-4425-aafa-b4b0471ae784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:05:59.939402 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:00.034605 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-3f6c0e09-cf0c-4425-aafa-b4b0471ae784 None None] MFA Rules not processed for user `91267a049343452eb208205ee3ea27c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:00.036427 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-3f6c0e09-cf0c-4425-aafa-b4b0471ae784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:00.227722 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 674/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Aug 23 06:05:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:00.240423 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458e031-7222-4f6a-9563-7805fc54b41c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:00.243027 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9458e031-7222-4f6a-9563-7805fc54b41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:00.428500 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458e031-7222-4f6a-9563-7805fc54b41c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:00.513531 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:00.561251 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9287776-2d49-49d4-955b-7f3a86710d6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:00.562172 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9287776-2d49-49d4-955b-7f3a86710d6f service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:00.562642 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9287776-2d49-49d4-955b-7f3a86710d6f service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:00.563089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9287776-2d49-49d4-955b-7f3a86710d6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:00.564864 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b9287776-2d49-49d4-955b-7f3a86710d6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:00.664309 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9287776-2d49-49d4-955b-7f3a86710d6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:00.666250 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9287776-2d49-49d4-955b-7f3a86710d6f service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:00.689853 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 686/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:06:02.928212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b204c35-84a7-4443-a59a-06e10fc41d6e None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:02.928717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b204c35-84a7-4443-a59a-06e10fc41d6e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:02.928717 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b204c35-84a7-4443-a59a-06e10fc41d6e None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:02.930613 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 675/1361] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:02.936080 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553fb43-eece-4a63-b36d-a295c5efa470 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:02.936587 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553fb43-eece-4a63-b36d-a295c5efa470 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:02.936680 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553fb43-eece-4a63-b36d-a295c5efa470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:03.039053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-6553fb43-eece-4a63-b36d-a295c5efa470 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:03.040847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6553fb43-eece-4a63-b36d-a295c5efa470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:03.086710 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:03.165781 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 687/1362] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:03.513470 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b204c35-84a7-4443-a59a-06e10fc41d6e None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:03.515298 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6b204c35-84a7-4443-a59a-06e10fc41d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:03.600584 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:03.700763 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b204c35-84a7-4443-a59a-06e10fc41d6e None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:03.841890 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82317599-ac72-493e-a7a5-9ee450d98202 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:03.842772 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82317599-ac72-493e-a7a5-9ee450d98202 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:03.843102 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82317599-ac72-493e-a7a5-9ee450d98202 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:03.843477 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82317599-ac72-493e-a7a5-9ee450d98202 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:03.845706 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-82317599-ac72-493e-a7a5-9ee450d98202 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:03.876169 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:03.958406 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82317599-ac72-493e-a7a5-9ee450d98202 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:03.960017 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82317599-ac72-493e-a7a5-9ee450d98202 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:03.982083 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 676/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:06:06.353863 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4ece8-6029-4822-a2a4-8ba8b91b60bd None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:06.354422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4ece8-6029-4822-a2a4-8ba8b91b60bd None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:06.354422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4ece8-6029-4822-a2a4-8ba8b91b60bd None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:06.355779 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 688/1364] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:06.362263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663abb9-9c0d-46c0-a940-234cdd53ea1f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:06.362742 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663abb9-9c0d-46c0-a940-234cdd53ea1f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:06.362862 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663abb9-9c0d-46c0-a940-234cdd53ea1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:06.436070 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-9663abb9-9c0d-46c0-a940-234cdd53ea1f None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:06.437565 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9663abb9-9c0d-46c0-a940-234cdd53ea1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:06.615807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:06.627817 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 677/1365] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:06.915184 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd8dc1-f158-484f-9b5e-c803ecb4e2b7 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:06.915464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd8dc1-f158-484f-9b5e-c803ecb4e2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:06.915738 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd8dc1-f158-484f-9b5e-c803ecb4e2b7 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:06.917067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 689/1366] 10.222.0.20 () {54 vars in 962 bytes} [Tue Aug 23 06:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:06.922802 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1294b-ae29-490f-a586-591ca8e90c40 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:06.923091 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1294b-ae29-490f-a586-591ca8e90c40 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:06.923533 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1294b-ae29-490f-a586-591ca8e90c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:07.068570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-23a1294b-ae29-490f-a586-591ca8e90c40 None None] MFA Rules not processed for user `91267a049343452eb208205ee3ea27c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:07.069818 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-23a1294b-ae29-490f-a586-591ca8e90c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:07.240886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 678/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Aug 23 06:06:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:07.250008 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd8dc1-f158-484f-9b5e-c803ecb4e2b7 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:07.252396 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-37cd8dc1-f158-484f-9b5e-c803ecb4e2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:07.504317 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd8dc1-f158-484f-9b5e-c803ecb4e2b7 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:07.532116 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:07.666895 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a46ac3-187f-45aa-a7d0-cf25649cff51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:07.668067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a46ac3-187f-45aa-a7d0-cf25649cff51 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:07.668593 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a46ac3-187f-45aa-a7d0-cf25649cff51 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:07.669081 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a46ac3-187f-45aa-a7d0-cf25649cff51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:07.671185 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-30a46ac3-187f-45aa-a7d0-cf25649cff51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:07.784188 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a46ac3-187f-45aa-a7d0-cf25649cff51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:07.786318 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a46ac3-187f-45aa-a7d0-cf25649cff51 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:07.823325 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 690/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:06:09.276427 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28db52-75c6-4ebb-8576-98c59f53b85d None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:09.276996 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28db52-75c6-4ebb-8576-98c59f53b85d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:09.276996 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28db52-75c6-4ebb-8576-98c59f53b85d None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:09.283853 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2f713-7de7-4f11-8bb9-6bb7fed637c6 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:09.284304 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2f713-7de7-4f11-8bb9-6bb7fed637c6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:09.284405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2f713-7de7-4f11-8bb9-6bb7fed637c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:09.286809 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 679/1369] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:06:09 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:09.322336 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:09.368159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ecd2f713-7de7-4f11-8bb9-6bb7fed637c6 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:09.369588 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ecd2f713-7de7-4f11-8bb9-6bb7fed637c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:09.503427 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 691/1370] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 06:06:09.880009 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77f302-2414-496f-a87a-cfa8532f1387 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:09.880286 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77f302-2414-496f-a87a-cfa8532f1387 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:09.880621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77f302-2414-496f-a87a-cfa8532f1387 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:09.887260 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 680/1371] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:06:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:09.888904 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a015f0d-ed2c-41c0-91c2-922b1dd2de8b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:09.889192 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a015f0d-ed2c-41c0-91c2-922b1dd2de8b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:09.889458 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a015f0d-ed2c-41c0-91c2-922b1dd2de8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:09.976819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-1a015f0d-ed2c-41c0-91c2-922b1dd2de8b None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:09.980225 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1a015f0d-ed2c-41c0-91c2-922b1dd2de8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:10.047469 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:10.115718 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 692/1372] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:10.123767 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ab6f4-ee2d-462b-8d2e-8646328cff4f None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:10.124045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ab6f4-ee2d-462b-8d2e-8646328cff4f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:10.124323 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ab6f4-ee2d-462b-8d2e-8646328cff4f None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:10.126178 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 681/1373] 10.222.0.20 () {54 vars in 964 bytes} [Tue Aug 23 06:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:10.131330 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988cd12-9ae8-40cc-8078-9a9720f70d48 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:10.131566 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988cd12-9ae8-40cc-8078-9a9720f70d48 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:10.131887 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988cd12-9ae8-40cc-8078-9a9720f70d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:10.220953 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-7988cd12-9ae8-40cc-8078-9a9720f70d48 None None] MFA Rules not processed for user `c741b92684794c338d63913774ab841a`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:10.222432 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7988cd12-9ae8-40cc-8078-9a9720f70d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:10.318377 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 693/1374] 10.222.0.20 () {58 vars in 1061 bytes} [Tue Aug 23 06:06:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:10.328183 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ab6f4-ee2d-462b-8d2e-8646328cff4f None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:10.329721 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-555ab6f4-ee2d-462b-8d2e-8646328cff4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:10.390637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:10.412383 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988cd12-9ae8-40cc-8078-9a9720f70d48 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:10.413647 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7988cd12-9ae8-40cc-8078-9a9720f70d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:10.459216 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ab6f4-ee2d-462b-8d2e-8646328cff4f None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:10.476918 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:10.547690 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f6075-e14e-47fd-93a3-b0b86d1c84fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c741b92684794c338d63913774ab841a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:10.549214 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f6075-e14e-47fd-93a3-b0b86d1c84fe service cinder] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:10.549741 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f6075-e14e-47fd-93a3-b0b86d1c84fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:10.550225 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f6075-e14e-47fd-93a3-b0b86d1c84fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:10.552118 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-633f6075-e14e-47fd-93a3-b0b86d1c84fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:10.560723 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988cd12-9ae8-40cc-8078-9a9720f70d48 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:10.670177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:10.687357 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f6075-e14e-47fd-93a3-b0b86d1c84fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:10.688818 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f6075-e14e-47fd-93a3-b0b86d1c84fe service cinder] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:10.709609 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 682/1375] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:06:10.715861 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067172e-be51-4ef0-b1a1-78710fe3a080 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:10.716550 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067172e-be51-4ef0-b1a1-78710fe3a080 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:10.716894 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067172e-be51-4ef0-b1a1-78710fe3a080 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:10.717249 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067172e-be51-4ef0-b1a1-78710fe3a080 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:10.718943 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-8067172e-be51-4ef0-b1a1-78710fe3a080 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:10.764561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:10.823580 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067172e-be51-4ef0-b1a1-78710fe3a080 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:10.827025 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067172e-be51-4ef0-b1a1-78710fe3a080 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:10.853826 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 694/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:06:12.911668 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5ff8b-8529-4849-bec5-28c327b8c060 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:12.912542 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5ff8b-8529-4849-bec5-28c327b8c060 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:12.912823 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5ff8b-8529-4849-bec5-28c327b8c060 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:12.915096 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 683/1377] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:06:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:06:12.920561 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc964e3-3140-4d4f-ad27-40a72e8a293e None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:12.921053 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc964e3-3140-4d4f-ad27-40a72e8a293e None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:12.921137 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc964e3-3140-4d4f-ad27-40a72e8a293e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:13.014783 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-2bc964e3-3140-4d4f-ad27-40a72e8a293e None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:13.016577 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2bc964e3-3140-4d4f-ad27-40a72e8a293e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:13.154645 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 695/1378] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:13.450193 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5ff8b-8529-4849-bec5-28c327b8c060 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:13.451760 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ece5ff8b-8529-4849-bec5-28c327b8c060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:13.661869 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5ff8b-8529-4849-bec5-28c327b8c060 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:13.732330 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:13.848949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433e4a7-9ac8-438a-b37e-f1b7ee05291a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:13.849568 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433e4a7-9ac8-438a-b37e-f1b7ee05291a service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:13.849846 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433e4a7-9ac8-438a-b37e-f1b7ee05291a service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:13.850116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433e4a7-9ac8-438a-b37e-f1b7ee05291a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:13.852597 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b433e4a7-9ac8-438a-b37e-f1b7ee05291a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:13.956069 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433e4a7-9ac8-438a-b37e-f1b7ee05291a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:13.957716 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433e4a7-9ac8-438a-b37e-f1b7ee05291a service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:13.978979 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 684/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:06:16.107600 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705afad-789e-4fd6-ad2f-716f7e1d65e3 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:16.108150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705afad-789e-4fd6-ad2f-716f7e1d65e3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:16.108150 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705afad-789e-4fd6-ad2f-716f7e1d65e3 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:16.114893 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e1e8e-1553-40e6-9c36-5402c791d26e None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:16.115314 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e1e8e-1553-40e6-9c36-5402c791d26e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:16.115514 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e1e8e-1553-40e6-9c36-5402c791d26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:16.120830 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 696/1380] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:06:16 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:16.188703 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-2f3e1e8e-1553-40e6-9c36-5402c791d26e None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:16.189911 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2f3e1e8e-1553-40e6-9c36-5402c791d26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:16.205026 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:16.303639 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 685/1381] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 06:06:16.588182 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705afad-789e-4fd6-ad2f-716f7e1d65e3 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:16.590002 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d705afad-789e-4fd6-ad2f-716f7e1d65e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:16.611212 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:16.766141 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705afad-789e-4fd6-ad2f-716f7e1d65e3 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:16.891567 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:16.910995 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86499704-9e8d-48bd-bdaa-37739854b33d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:16.911973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86499704-9e8d-48bd-bdaa-37739854b33d service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:16.912437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86499704-9e8d-48bd-bdaa-37739854b33d service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:16.912890 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86499704-9e8d-48bd-bdaa-37739854b33d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:16.915535 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-86499704-9e8d-48bd-bdaa-37739854b33d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:17.028379 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86499704-9e8d-48bd-bdaa-37739854b33d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:17.030965 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86499704-9e8d-48bd-bdaa-37739854b33d service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:17.052722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 697/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:06:19.093614 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885aaf9-fad0-4fb0-96a9-5c0147fe25db None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:19.094207 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885aaf9-fad0-4fb0-96a9-5c0147fe25db None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:19.094207 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885aaf9-fad0-4fb0-96a9-5c0147fe25db None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:19.104801 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 686/1383] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:06:19 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:19.107583 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913f298-e1f0-4061-a46b-da2f23121fd0 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:19.108139 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913f298-e1f0-4061-a46b-da2f23121fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:19.108241 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913f298-e1f0-4061-a46b-da2f23121fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:19.197168 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-c913f298-e1f0-4061-a46b-da2f23121fd0 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:19.198429 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c913f298-e1f0-4061-a46b-da2f23121fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:19.263864 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:19.337852 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 698/1384] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:19.659620 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885aaf9-fad0-4fb0-96a9-5c0147fe25db None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:19.661771 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d885aaf9-fad0-4fb0-96a9-5c0147fe25db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:19.837052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:19.841795 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885aaf9-fad0-4fb0-96a9-5c0147fe25db None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:19.989258 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2badaf-0c80-4fdf-a6d0-5c7eb34ca849 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:19.989872 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2badaf-0c80-4fdf-a6d0-5c7eb34ca849 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:19.990147 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2badaf-0c80-4fdf-a6d0-5c7eb34ca849 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:19.990414 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2badaf-0c80-4fdf-a6d0-5c7eb34ca849 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:19.992696 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1d2badaf-0c80-4fdf-a6d0-5c7eb34ca849 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:20.094951 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2badaf-0c80-4fdf-a6d0-5c7eb34ca849 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:20.096520 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2badaf-0c80-4fdf-a6d0-5c7eb34ca849 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:20.118413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 687/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:06:22.189503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597f54d-0f80-4d2c-9151-445f0dd204cb None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:22.190087 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597f54d-0f80-4d2c-9151-445f0dd204cb None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:22.190087 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597f54d-0f80-4d2c-9151-445f0dd204cb None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:22.191686 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 699/1386] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:22.198469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87d759-8122-43f9-ad7a-3635a6a49567 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:22.198982 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87d759-8122-43f9-ad7a-3635a6a49567 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:22.199185 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87d759-8122-43f9-ad7a-3635a6a49567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:22.205454 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:22.277963 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-4a87d759-8122-43f9-ad7a-3635a6a49567 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:22.279220 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4a87d759-8122-43f9-ad7a-3635a6a49567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:22.406469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 688/1387] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:22.708596 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597f54d-0f80-4d2c-9151-445f0dd204cb None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:22.710719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e597f54d-0f80-4d2c-9151-445f0dd204cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:22.884959 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597f54d-0f80-4d2c-9151-445f0dd204cb None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:22.932482 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:23.055685 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fffe9-f61c-406e-9120-96a7196a2840 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:23.056444 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fffe9-f61c-406e-9120-96a7196a2840 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:23.056762 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fffe9-f61c-406e-9120-96a7196a2840 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:23.057023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fffe9-f61c-406e-9120-96a7196a2840 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:23.058722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-bb4fffe9-f61c-406e-9120-96a7196a2840 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:23.169072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4fffe9-f61c-406e-9120-96a7196a2840 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:23.170887 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4fffe9-f61c-406e-9120-96a7196a2840 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:23.191387 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 700/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:06:25.242829 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312e4ab-e8ba-4bc0-86db-14a89c9ae30a None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:25.243393 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312e4ab-e8ba-4bc0-86db-14a89c9ae30a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:25.243393 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312e4ab-e8ba-4bc0-86db-14a89c9ae30a None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:25.249747 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 689/1389] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:06:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:25.250411 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e4408-2560-4537-9121-0c679383d2e8 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:25.251149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e4408-2560-4537-9121-0c679383d2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:25.251262 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e4408-2560-4537-9121-0c679383d2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:25.321507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:25.338907 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-845e4408-2560-4537-9121-0c679383d2e8 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:25.340669 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-845e4408-2560-4537-9121-0c679383d2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:25.450253 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 701/1390] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 06:06:25.739810 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312e4ab-e8ba-4bc0-86db-14a89c9ae30a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:25.741957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e312e4ab-e8ba-4bc0-86db-14a89c9ae30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:25.841574 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:25.931168 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312e4ab-e8ba-4bc0-86db-14a89c9ae30a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:26.070006 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45e587-158f-4486-acc9-1f9f38104a00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:26.070647 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45e587-158f-4486-acc9-1f9f38104a00 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:26.070922 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45e587-158f-4486-acc9-1f9f38104a00 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:26.071192 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45e587-158f-4486-acc9-1f9f38104a00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:26.072857 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-ae45e587-158f-4486-acc9-1f9f38104a00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:26.125930 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:26.183211 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45e587-158f-4486-acc9-1f9f38104a00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:26.184850 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45e587-158f-4486-acc9-1f9f38104a00 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:26.204163 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 690/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:06:27.133404 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293578-a8cb-440c-9693-fc900a948e13 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:27.133980 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293578-a8cb-440c-9693-fc900a948e13 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:27.134256 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293578-a8cb-440c-9693-fc900a948e13 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:27.136522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 702/1392] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:06:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:27.143805 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107de16-6f99-4aac-badd-69728d588281 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:27.144282 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107de16-6f99-4aac-badd-69728d588281 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:27.144371 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107de16-6f99-4aac-badd-69728d588281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:27.224877 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-1107de16-6f99-4aac-badd-69728d588281 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:27.226604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1107de16-6f99-4aac-badd-69728d588281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:27.342297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 691/1393] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 06:06:27.658337 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bf359-7225-402e-b4e0-e8ceae76c60f None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:27.658725 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bf359-7225-402e-b4e0-e8ceae76c60f None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:27.658931 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bf359-7225-402e-b4e0-e8ceae76c60f None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:27.660408 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 703/1394] 10.222.0.20 () {54 vars in 967 bytes} [Tue Aug 23 06:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:27.665328 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47774444-99a1-469b-a090-8c1b927782e3 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:27.665617 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47774444-99a1-469b-a090-8c1b927782e3 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:27.665892 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47774444-99a1-469b-a090-8c1b927782e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:27.695114 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:27.753257 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-47774444-99a1-469b-a090-8c1b927782e3 None None] MFA Rules not processed for user `89621d839d1c44798f691db4d49e1278`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:27.754563 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-47774444-99a1-469b-a090-8c1b927782e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:27.848008 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 692/1395] 10.222.0.20 () {58 vars in 1064 bytes} [Tue Aug 23 06:06:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:27.855815 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bf359-7225-402e-b4e0-e8ceae76c60f None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:27.857485 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e96bf359-7225-402e-b4e0-e8ceae76c60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:27.975668 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bf359-7225-402e-b4e0-e8ceae76c60f None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:27.998824 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:28.059024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0843ec-3c66-4180-8488-56a8a1c8385c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89621d839d1c44798f691db4d49e1278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:28.059545 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0843ec-3c66-4180-8488-56a8a1c8385c service glance] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:28.059823 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0843ec-3c66-4180-8488-56a8a1c8385c service glance] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:28.060089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0843ec-3c66-4180-8488-56a8a1c8385c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:28.062405 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5a0843ec-3c66-4180-8488-56a8a1c8385c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:28.183331 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0843ec-3c66-4180-8488-56a8a1c8385c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:28.184771 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0843ec-3c66-4180-8488-56a8a1c8385c service glance] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:28.200285 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe4915-1792-4752-b2ce-bd411e4be380 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:28.200806 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe4915-1792-4752-b2ce-bd411e4be380 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:28.200806 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe4915-1792-4752-b2ce-bd411e4be380 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:28.202565 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 693/1396] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 06:06:28.203703 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 704/1397] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:06:28.219949 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce673fb-9f4c-4d9a-9cce-2d476847454f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:28.220224 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce673fb-9f4c-4d9a-9cce-2d476847454f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:28.220492 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce673fb-9f4c-4d9a-9cce-2d476847454f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:28.300950 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-cce673fb-9f4c-4d9a-9cce-2d476847454f None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:28.302167 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-cce673fb-9f4c-4d9a-9cce-2d476847454f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:28.377864 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:28.426284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 694/1398] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:28.741854 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0843ec-3c66-4180-8488-56a8a1c8385c service glance] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:28.743070 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5a0843ec-3c66-4180-8488-56a8a1c8385c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:28.807288 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:28.863913 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0843ec-3c66-4180-8488-56a8a1c8385c service glance] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:29.024233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fa703-b459-4cc4-ae89-93f15f7d096a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:29.024517 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fa703-b459-4cc4-ae89-93f15f7d096a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:29.024822 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fa703-b459-4cc4-ae89-93f15f7d096a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:29.033977 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b21695-10bf-45cb-a995-f61e78e5a38e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:29.034746 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b21695-10bf-45cb-a995-f61e78e5a38e service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:29.035089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b21695-10bf-45cb-a995-f61e78e5a38e service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:29.035453 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b21695-10bf-45cb-a995-f61e78e5a38e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:29.038799 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-75b21695-10bf-45cb-a995-f61e78e5a38e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:29.101967 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-3b3fa703-b459-4cc4-ae89-93f15f7d096a None None] MFA Rules not processed for user `078a012fea464954b0632a6ce86d7cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:29.103346 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-3b3fa703-b459-4cc4-ae89-93f15f7d096a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:29.120138 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:29.153948 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b21695-10bf-45cb-a995-f61e78e5a38e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:29.155750 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b21695-10bf-45cb-a995-f61e78e5a38e service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:29.176274 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 705/1399] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:06:29.211064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 695/1400] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Aug 23 06:06:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 06:06:29.265350 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd127e5-3f39-4c85-910a-e3463099240c None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:29.265874 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd127e5-3f39-4c85-910a-e3463099240c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:29.266048 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd127e5-3f39-4c85-910a-e3463099240c None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:29.267745 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 706/1401] 10.222.0.20 () {54 vars in 943 bytes} [Tue Aug 23 06:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:29.276009 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eba872-aaf5-4d85-8c71-0c532806c4db None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:29.276318 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eba872-aaf5-4d85-8c71-0c532806c4db None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:29.276551 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eba872-aaf5-4d85-8c71-0c532806c4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:29.364506 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:29.373557 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-51eba872-aaf5-4d85-8c71-0c532806c4db None None] MFA Rules not processed for user `a73720a950944fa08bf1276ea78423b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:29.374807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-51eba872-aaf5-4d85-8c71-0c532806c4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:29.517707 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 696/1402] 10.222.0.20 () {58 vars in 1040 bytes} [Tue Aug 23 06:06:29 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:29.518950 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd127e5-3f39-4c85-910a-e3463099240c None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:29.520730 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-fbd127e5-3f39-4c85-910a-e3463099240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:29.695372 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd127e5-3f39-4c85-910a-e3463099240c None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:29.761819 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:29.818387 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d4b90-f626-4c28-860e-3c0c0ecea5da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a73720a950944fa08bf1276ea78423b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:29.819159 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d4b90-f626-4c28-860e-3c0c0ecea5da service swift] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:29.819420 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d4b90-f626-4c28-860e-3c0c0ecea5da service swift] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:29.819684 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d4b90-f626-4c28-860e-3c0c0ecea5da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:29.821475 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7b9d4b90-f626-4c28-860e-3c0c0ecea5da service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:29.916927 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d4b90-f626-4c28-860e-3c0c0ecea5da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:29.918614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d4b90-f626-4c28-860e-3c0c0ecea5da service swift] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:29.928289 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 707/1403] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Aug 23 06:06:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 06:06:31.399459 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7b350-0fdb-4a15-94d7-f9926a7b79ca None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:31.400019 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7b350-0fdb-4a15-94d7-f9926a7b79ca None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:31.400019 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7b350-0fdb-4a15-94d7-f9926a7b79ca None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:31.401376 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 697/1404] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:31.407501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853fdd2c-0c11-46d4-bf27-b1d5c3897a6b None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:31.407501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853fdd2c-0c11-46d4-bf27-b1d5c3897a6b None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:31.408134 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853fdd2c-0c11-46d4-bf27-b1d5c3897a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:31.503246 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-853fdd2c-0c11-46d4-bf27-b1d5c3897a6b None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:31.504958 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-853fdd2c-0c11-46d4-bf27-b1d5c3897a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:31.554349 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:31.655713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 708/1405] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:31.960848 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7b350-0fdb-4a15-94d7-f9926a7b79ca None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:31.966745 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-7be7b350-0fdb-4a15-94d7-f9926a7b79ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:32.129320 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:32.183267 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7b350-0fdb-4a15-94d7-f9926a7b79ca None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:32.357302 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a5f2d-ec95-4c8e-95e5-d5090edc1438 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:32.357928 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a5f2d-ec95-4c8e-95e5-d5090edc1438 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:32.358206 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a5f2d-ec95-4c8e-95e5-d5090edc1438 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:32.358704 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a5f2d-ec95-4c8e-95e5-d5090edc1438 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:32.360386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-bb2a5f2d-ec95-4c8e-95e5-d5090edc1438 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:32.473125 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:32.490128 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a5f2d-ec95-4c8e-95e5-d5090edc1438 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:32.492518 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a5f2d-ec95-4c8e-95e5-d5090edc1438 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:32.528812 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 698/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:06:34.657749 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98fc3b-21f4-4938-970b-08cd6cbb4ae5 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:34.658326 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98fc3b-21f4-4938-970b-08cd6cbb4ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:34.658326 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98fc3b-21f4-4938-970b-08cd6cbb4ae5 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:34.664349 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 709/1407] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:06:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:34.666210 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572de3a-91c9-4d83-90e8-d70c3b270b90 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:34.667092 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572de3a-91c9-4d83-90e8-d70c3b270b90 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:34.667703 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572de3a-91c9-4d83-90e8-d70c3b270b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:34.760664 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-4572de3a-91c9-4d83-90e8-d70c3b270b90 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:34.762041 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4572de3a-91c9-4d83-90e8-d70c3b270b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:34.899860 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 699/1408] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 06:06:35.223536 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98fc3b-21f4-4938-970b-08cd6cbb4ae5 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:35.226639 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ea98fc3b-21f4-4938-970b-08cd6cbb4ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:35.428876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98fc3b-21f4-4938-970b-08cd6cbb4ae5 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:35.459770 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:35.602930 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051c37f-e025-4ca9-aee9-a3811f08dc26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:35.603554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051c37f-e025-4ca9-aee9-a3811f08dc26 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:35.603836 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051c37f-e025-4ca9-aee9-a3811f08dc26 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:35.604107 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051c37f-e025-4ca9-aee9-a3811f08dc26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:35.605788 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3051c37f-e025-4ca9-aee9-a3811f08dc26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:35.722963 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051c37f-e025-4ca9-aee9-a3811f08dc26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:35.725786 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051c37f-e025-4ca9-aee9-a3811f08dc26 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:35.751611 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 710/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:06:44.895013 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=76 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:45.748793 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=89 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:53.793926 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a2bca-788a-488f-bafc-8dad9b28602e None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:53.794476 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a2bca-788a-488f-bafc-8dad9b28602e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:53.794476 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a2bca-788a-488f-bafc-8dad9b28602e None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:53.795768 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 700/1410] 10.222.0.20 () {54 vars in 918 bytes} [Tue Aug 23 06:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:53.801101 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612064a8-9498-46a6-bb9c-73d56572e38c None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:53.801573 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612064a8-9498-46a6-bb9c-73d56572e38c None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:53.801699 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612064a8-9498-46a6-bb9c-73d56572e38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:53.886456 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-612064a8-9498-46a6-bb9c-73d56572e38c None None] MFA Rules not processed for user `0e16f51a2c714b2fb3d7875b589f9a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:53.887945 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-612064a8-9498-46a6-bb9c-73d56572e38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:54.010234 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 711/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Aug 23 06:06:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:54.024320 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fca99-e2ce-4f15-b501-38f5a62e0d2c None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:54.024604 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fca99-e2ce-4f15-b501-38f5a62e0d2c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:54.024882 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fca99-e2ce-4f15-b501-38f5a62e0d2c None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:54.026227 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 701/1412] 10.222.0.20 () {54 vars in 918 bytes} [Tue Aug 23 06:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:54.031069 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a278a8-47ba-44c2-8334-0c514a1e2068 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:54.031361 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a278a8-47ba-44c2-8334-0c514a1e2068 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:54.031632 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a278a8-47ba-44c2-8334-0c514a1e2068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:54.111395 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:54.115265 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-44a278a8-47ba-44c2-8334-0c514a1e2068 None None] MFA Rules not processed for user `91267a049343452eb208205ee3ea27c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:54.116811 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-44a278a8-47ba-44c2-8334-0c514a1e2068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:54.238966 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 712/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Aug 23 06:06:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:56.699573 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334d07c-5e56-4aa8-8bcb-45b13e54b72d None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:56.700252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334d07c-5e56-4aa8-8bcb-45b13e54b72d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:56.700252 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334d07c-5e56-4aa8-8bcb-45b13e54b72d None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:56.701447 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 06:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:56.707474 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c515c-9329-4192-aa2e-b4946d5af3b8 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:56.707994 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c515c-9329-4192-aa2e-b4946d5af3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:56.708097 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c515c-9329-4192-aa2e-b4946d5af3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:56.785601 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-929c515c-9329-4192-aa2e-b4946d5af3b8 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:56.787568 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-929c515c-9329-4192-aa2e-b4946d5af3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:56.850220 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:56.901112 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 06:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 06:06:57.205757 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334d07c-5e56-4aa8-8bcb-45b13e54b72d None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:57.207494 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-4334d07c-5e56-4aa8-8bcb-45b13e54b72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:57.376066 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334d07c-5e56-4aa8-8bcb-45b13e54b72d None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:57.453870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:57.498677 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e9eed-379f-494b-a6e4-4ad37da7b094 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:57.498677 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e9eed-379f-494b-a6e4-4ad37da7b094 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:57.498677 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e9eed-379f-494b-a6e4-4ad37da7b094 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:57.498677 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e9eed-379f-494b-a6e4-4ad37da7b094 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:57.499140 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-304e9eed-379f-494b-a6e4-4ad37da7b094 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:57.603760 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e9eed-379f-494b-a6e4-4ad37da7b094 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:57.605407 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e9eed-379f-494b-a6e4-4ad37da7b094 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:57.626007 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 703/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:06:58.678134 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80708abd-77a7-4a80-887e-77f9f460d955 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:58.679061 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80708abd-77a7-4a80-887e-77f9f460d955 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:58.679231 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80708abd-77a7-4a80-887e-77f9f460d955 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:58.680626 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 06:06:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:06:58.686400 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab303e-55fb-4a26-bef4-8d3de78f7f2f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:58.687043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab303e-55fb-4a26-bef4-8d3de78f7f2f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:58.687043 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab303e-55fb-4a26-bef4-8d3de78f7f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:58.775327 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-baab303e-55fb-4a26-bef4-8d3de78f7f2f None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:06:58.776565 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-baab303e-55fb-4a26-bef4-8d3de78f7f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:58.820431 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:58.894064 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 06:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:06:59.050729 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80708abd-77a7-4a80-887e-77f9f460d955 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:06:59.052878 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-80708abd-77a7-4a80-887e-77f9f460d955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:59.213851 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80708abd-77a7-4a80-887e-77f9f460d955 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:06:59.255519 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:06:59.334892 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa6d01-4f33-424d-b3ba-8e8670496159 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:06:59.335503 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa6d01-4f33-424d-b3ba-8e8670496159 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:06:59.335785 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa6d01-4f33-424d-b3ba-8e8670496159 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:06:59.336080 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa6d01-4f33-424d-b3ba-8e8670496159 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:06:59.337738 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b8fa6d01-4f33-424d-b3ba-8e8670496159 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:06:59.429596 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa6d01-4f33-424d-b3ba-8e8670496159 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:06:59.431220 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa6d01-4f33-424d-b3ba-8e8670496159 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:06:59.451123 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 715/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:00.416852 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920b10b-fd88-4f59-9bbb-c7ccbdef23c4 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:00.417426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920b10b-fd88-4f59-9bbb-c7ccbdef23c4 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:00.417426 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920b10b-fd88-4f59-9bbb-c7ccbdef23c4 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:00.418824 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 06:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:00.424241 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8be002-30d2-4a53-96e2-e6f1f9880a78 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:00.424732 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8be002-30d2-4a53-96e2-e6f1f9880a78 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:00.424838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8be002-30d2-4a53-96e2-e6f1f9880a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:00.487681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:00.513697 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-ac8be002-30d2-4a53-96e2-e6f1f9880a78 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:00.515072 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-ac8be002-30d2-4a53-96e2-e6f1f9880a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:00.634559 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 06:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 06:07:00.802918 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920b10b-fd88-4f59-9bbb-c7ccbdef23c4 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:00.804526 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6920b10b-fd88-4f59-9bbb-c7ccbdef23c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:00.989266 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920b10b-fd88-4f59-9bbb-c7ccbdef23c4 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:01.003158 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:01.118612 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9f21a-530c-472a-aca0-737545fdfa22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:01.119133 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9f21a-530c-472a-aca0-737545fdfa22 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:01.119405 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9f21a-530c-472a-aca0-737545fdfa22 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:01.119700 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9f21a-530c-472a-aca0-737545fdfa22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:01.121297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fed9f21a-530c-472a-aca0-737545fdfa22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:01.221685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9f21a-530c-472a-aca0-737545fdfa22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:01.223360 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9f21a-530c-472a-aca0-737545fdfa22 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:01.245465 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 706/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 06:07:02.133762 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809219c7-7a18-4220-8c75-f3742a473cd6 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:02.134294 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809219c7-7a18-4220-8c75-f3742a473cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:02.134294 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809219c7-7a18-4220-8c75-f3742a473cd6 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:02.135695 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 06:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:02.141009 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c80ead-6532-4e14-ab20-84e0cb6c0774 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:02.141429 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c80ead-6532-4e14-ab20-84e0cb6c0774 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:02.141539 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c80ead-6532-4e14-ab20-84e0cb6c0774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:02.161218 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:02.219874 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-c3c80ead-6532-4e14-ab20-84e0cb6c0774 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:02.220990 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-c3c80ead-6532-4e14-ab20-84e0cb6c0774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:02.322376 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 06:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:02.476496 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809219c7-7a18-4220-8c75-f3742a473cd6 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:02.477967 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-809219c7-7a18-4220-8c75-f3742a473cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:02.591006 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:02.628683 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809219c7-7a18-4220-8c75-f3742a473cd6 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:02.748429 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d6dac-14fc-4afd-a6d6-0b15d2b7f10f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:02.748963 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d6dac-14fc-4afd-a6d6-0b15d2b7f10f service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:02.749243 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d6dac-14fc-4afd-a6d6-0b15d2b7f10f service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:02.749514 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d6dac-14fc-4afd-a6d6-0b15d2b7f10f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:02.751409 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3c2d6dac-14fc-4afd-a6d6-0b15d2b7f10f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:02.832043 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:02.846885 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d6dac-14fc-4afd-a6d6-0b15d2b7f10f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:02.848515 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d6dac-14fc-4afd-a6d6-0b15d2b7f10f service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:02.868007 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 718/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:03.740973 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d9b28-b0aa-41d9-a2d5-0579b8445eb9 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:03.741535 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d9b28-b0aa-41d9-a2d5-0579b8445eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:03.741535 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d9b28-b0aa-41d9-a2d5-0579b8445eb9 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:03.742979 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 06:07:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:03.748181 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b573a-d07f-47f5-8f00-98381046b9d7 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:03.748622 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b573a-d07f-47f5-8f00-98381046b9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:03.748738 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b573a-d07f-47f5-8f00-98381046b9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:03.826505 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-a97b573a-d07f-47f5-8f00-98381046b9d7 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:03.827651 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a97b573a-d07f-47f5-8f00-98381046b9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:03.939136 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 06:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:04.099530 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d9b28-b0aa-41d9-a2d5-0579b8445eb9 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:04.101218 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-0a3d9b28-b0aa-41d9-a2d5-0579b8445eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:04.188951 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:04.253243 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d9b28-b0aa-41d9-a2d5-0579b8445eb9 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:04.379171 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc11d4-f7f9-45a2-8d12-13d69cb7e886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:04.379693 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc11d4-f7f9-45a2-8d12-13d69cb7e886 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:04.380532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc11d4-f7f9-45a2-8d12-13d69cb7e886 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:04.380927 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc11d4-f7f9-45a2-8d12-13d69cb7e886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:04.382548 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6ebc11d4-f7f9-45a2-8d12-13d69cb7e886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:04.432181 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:04.477451 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc11d4-f7f9-45a2-8d12-13d69cb7e886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:04.479091 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc11d4-f7f9-45a2-8d12-13d69cb7e886 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:04.497317 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 709/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:05.331145 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e8b43-a5fa-40aa-ad9a-7a57f132fbbc None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:05.331722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e8b43-a5fa-40aa-ad9a-7a57f132fbbc None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:05.331722 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e8b43-a5fa-40aa-ad9a-7a57f132fbbc None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:05.335817 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 06:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:05.339524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713949a0-2a86-42c2-9ef0-6114d8c583a6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:05.340003 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713949a0-2a86-42c2-9ef0-6114d8c583a6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:05.340190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713949a0-2a86-42c2-9ef0-6114d8c583a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:05.425184 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-713949a0-2a86-42c2-9ef0-6114d8c583a6 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:05.426473 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-713949a0-2a86-42c2-9ef0-6114d8c583a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:05.528948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 06:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:05.683321 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e8b43-a5fa-40aa-ad9a-7a57f132fbbc None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:05.684876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-515e8b43-a5fa-40aa-ad9a-7a57f132fbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:05.736148 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:05.836606 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e8b43-a5fa-40aa-ad9a-7a57f132fbbc None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:05.957920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e7296-5616-419e-97f4-626b04b11161 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:05.959177 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e7296-5616-419e-97f4-626b04b11161 service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:05.959451 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e7296-5616-419e-97f4-626b04b11161 service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:05.959719 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e7296-5616-419e-97f4-626b04b11161 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:05.961365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-f28e7296-5616-419e-97f4-626b04b11161 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:05.969012 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:06.061806 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e7296-5616-419e-97f4-626b04b11161 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:06.063446 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e7296-5616-419e-97f4-626b04b11161 service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:06.082790 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 721/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:06.913263 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644edc5-079e-4eaf-be51-06755a7fceaa None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:06.913857 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644edc5-079e-4eaf-be51-06755a7fceaa None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:06.914028 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644edc5-079e-4eaf-be51-06755a7fceaa None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:06.915445 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 06:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:06.922042 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01192cf-504b-4fea-a9a9-3cd2ea3fb2e3 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:06.922579 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01192cf-504b-4fea-a9a9-3cd2ea3fb2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:06.922694 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01192cf-504b-4fea-a9a9-3cd2ea3fb2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:07.002887 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-d01192cf-504b-4fea-a9a9-3cd2ea3fb2e3 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:07.004093 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-d01192cf-504b-4fea-a9a9-3cd2ea3fb2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:07.075973 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:07.105789 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 06:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:07.258418 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644edc5-079e-4eaf-be51-06755a7fceaa None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:07.259981 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-6644edc5-079e-4eaf-be51-06755a7fceaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:07.279015 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:07.420405 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644edc5-079e-4eaf-be51-06755a7fceaa None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:07.527525 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:07.558566 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c18a35-16e1-465b-bbd2-f7d5ab431804 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:07.559191 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c18a35-16e1-465b-bbd2-f7d5ab431804 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:07.559475 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c18a35-16e1-465b-bbd2-f7d5ab431804 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:07.559736 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c18a35-16e1-465b-bbd2-f7d5ab431804 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:07.561362 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-51c18a35-16e1-465b-bbd2-f7d5ab431804 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:07.659866 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c18a35-16e1-465b-bbd2-f7d5ab431804 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:07.662128 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c18a35-16e1-465b-bbd2-f7d5ab431804 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:07.680464 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 712/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:08.503666 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539b9c1-6b76-4381-85e5-7542cc8690f2 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:08.504243 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539b9c1-6b76-4381-85e5-7542cc8690f2 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:08.504243 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539b9c1-6b76-4381-85e5-7542cc8690f2 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:08.505723 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 06:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:08.511072 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a15c2b-bd24-469c-9369-6a9650b654cf None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:08.511564 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a15c2b-bd24-469c-9369-6a9650b654cf None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:08.511703 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a15c2b-bd24-469c-9369-6a9650b654cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:08.593863 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-91a15c2b-bd24-469c-9369-6a9650b654cf None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:08.595161 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-91a15c2b-bd24-469c-9369-6a9650b654cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:08.639584 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:08.703398 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 06:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:08.857970 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539b9c1-6b76-4381-85e5-7542cc8690f2 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:08.859440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e539b9c1-6b76-4381-85e5-7542cc8690f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:09.023627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539b9c1-6b76-4381-85e5-7542cc8690f2 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:09.083214 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:09.144464 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec60648-d324-4433-97ce-e5933a0e089e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:09.144975 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec60648-d324-4433-97ce-e5933a0e089e service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:09.145248 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec60648-d324-4433-97ce-e5933a0e089e service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:09.146212 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec60648-d324-4433-97ce-e5933a0e089e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:09.147882 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6ec60648-d324-4433-97ce-e5933a0e089e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:09.238672 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec60648-d324-4433-97ce-e5933a0e089e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:09.240332 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec60648-d324-4433-97ce-e5933a0e089e service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:09.260440 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 724/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:10.080670 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71831df8-af6b-45f6-8015-6622b578cce6 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:10.081232 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71831df8-af6b-45f6-8015-6622b578cce6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:10.081232 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71831df8-af6b-45f6-8015-6622b578cce6 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:10.082599 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 06:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:10.089556 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cce40e-bf19-4df9-a1d9-d1a8c893f369 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:10.090205 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cce40e-bf19-4df9-a1d9-d1a8c893f369 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:10.090205 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cce40e-bf19-4df9-a1d9-d1a8c893f369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:10.173330 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-e4cce40e-bf19-4df9-a1d9-d1a8c893f369 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:10.174609 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-e4cce40e-bf19-4df9-a1d9-d1a8c893f369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:10.177140 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:10.288559 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 06:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:10.448154 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71831df8-af6b-45f6-8015-6622b578cce6 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:10.449692 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-71831df8-af6b-45f6-8015-6622b578cce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:10.609187 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71831df8-af6b-45f6-8015-6622b578cce6 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:10.638381 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:10.730634 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865b507-d87f-4358-a0b3-eab9bba2a8d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:10.731239 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865b507-d87f-4358-a0b3-eab9bba2a8d7 service nova] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:10.731514 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865b507-d87f-4358-a0b3-eab9bba2a8d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:10.731781 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865b507-d87f-4358-a0b3-eab9bba2a8d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:10.733364 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-f865b507-d87f-4358-a0b3-eab9bba2a8d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:10.826442 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865b507-d87f-4358-a0b3-eab9bba2a8d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:10.828084 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865b507-d87f-4358-a0b3-eab9bba2a8d7 service nova] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:10.846636 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 715/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:12.710375 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b9898-4378-4ddc-8821-d50a6b351631 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:12.711002 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b9898-4378-4ddc-8821-d50a6b351631 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:12.711002 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b9898-4378-4ddc-8821-d50a6b351631 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:12.712304 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 06:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:12.718716 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5361f-04a4-42f7-b7b8-3965c011400f None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:12.719296 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5361f-04a4-42f7-b7b8-3965c011400f None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:12.719296 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5361f-04a4-42f7-b7b8-3965c011400f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:12.761393 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:12.794646 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-d8e5361f-04a4-42f7-b7b8-3965c011400f None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:12.795900 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d8e5361f-04a4-42f7-b7b8-3965c011400f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:12.897311 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 06:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:13.189915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b9898-4378-4ddc-8821-d50a6b351631 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:13.191485 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0a4b9898-4378-4ddc-8821-d50a6b351631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:13.299241 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b9898-4378-4ddc-8821-d50a6b351631 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:13.334363 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:13.379532 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35310b32-aabd-4bc7-b8df-ee04a2c9d4e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:13.380166 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35310b32-aabd-4bc7-b8df-ee04a2c9d4e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:13.380578 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35310b32-aabd-4bc7-b8df-ee04a2c9d4e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:13.380852 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35310b32-aabd-4bc7-b8df-ee04a2c9d4e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:13.382627 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-35310b32-aabd-4bc7-b8df-ee04a2c9d4e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:13.502912 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35310b32-aabd-4bc7-b8df-ee04a2c9d4e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:13.505572 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35310b32-aabd-4bc7-b8df-ee04a2c9d4e9 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:13.525302 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 727/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:15.608336 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234dec13-ea6c-4692-84a4-9a5806787c4b None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:15.608862 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234dec13-ea6c-4692-84a4-9a5806787c4b None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:15.608862 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234dec13-ea6c-4692-84a4-9a5806787c4b None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:15.610133 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 06:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:15.616205 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2ae16-d4c4-4362-9334-1b56e58776a8 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:15.616636 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2ae16-d4c4-4362-9334-1b56e58776a8 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:15.616748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2ae16-d4c4-4362-9334-1b56e58776a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:15.687402 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-6af2ae16-d4c4-4362-9334-1b56e58776a8 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:15.688621 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6af2ae16-d4c4-4362-9334-1b56e58776a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:15.702098 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:15.794371 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 06:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:16.099076 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234dec13-ea6c-4692-84a4-9a5806787c4b None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:16.100643 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-234dec13-ea6c-4692-84a4-9a5806787c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:16.212467 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234dec13-ea6c-4692-84a4-9a5806787c4b None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:16.225488 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:16.303499 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34624ae6-6619-407e-ab9e-74618d2f68d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:16.303499 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34624ae6-6619-407e-ab9e-74618d2f68d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:16.303499 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34624ae6-6619-407e-ab9e-74618d2f68d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:16.303499 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34624ae6-6619-407e-ab9e-74618d2f68d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:16.304764 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-34624ae6-6619-407e-ab9e-74618d2f68d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:16.422334 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34624ae6-6619-407e-ab9e-74618d2f68d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:16.424134 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34624ae6-6619-407e-ab9e-74618d2f68d2 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:16.445347 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 718/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:18.787278 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdcb13-2029-4ccf-90fa-7f14940c2903 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:18.787845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdcb13-2029-4ccf-90fa-7f14940c2903 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:18.787845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdcb13-2029-4ccf-90fa-7f14940c2903 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:18.789336 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 06:07:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:18.794637 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f9a36-cc61-4ecf-aa9f-8dcb987bb7b5 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:18.795044 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f9a36-cc61-4ecf-aa9f-8dcb987bb7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:18.795205 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f9a36-cc61-4ecf-aa9f-8dcb987bb7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:18.851841 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:18.872563 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-5b4f9a36-cc61-4ecf-aa9f-8dcb987bb7b5 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:18.873801 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5b4f9a36-cc61-4ecf-aa9f-8dcb987bb7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:18.998404 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 06:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:19.298186 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdcb13-2029-4ccf-90fa-7f14940c2903 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:19.299876 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-44bdcb13-2029-4ccf-90fa-7f14940c2903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:19.416118 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdcb13-2029-4ccf-90fa-7f14940c2903 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:19.459590 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:19.496550 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bbee0-a86d-4c50-ae13-e95008ae2ed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:19.497158 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bbee0-a86d-4c50-ae13-e95008ae2ed9 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:19.497437 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bbee0-a86d-4c50-ae13-e95008ae2ed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:19.497704 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bbee0-a86d-4c50-ae13-e95008ae2ed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:19.499367 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-246bbee0-a86d-4c50-ae13-e95008ae2ed9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:19.616556 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bbee0-a86d-4c50-ae13-e95008ae2ed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:19.618000 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bbee0-a86d-4c50-ae13-e95008ae2ed9 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:19.637820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 730/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 06:07:21.734704 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3124000-b3e8-4ad3-b817-02829e4d328c None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:21.735284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3124000-b3e8-4ad3-b817-02829e4d328c None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:21.735284 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3124000-b3e8-4ad3-b817-02829e4d328c None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:21.736774 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 06:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:21.742880 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b1c79-c46a-45b6-9435-824dbe994ad2 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:21.743442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b1c79-c46a-45b6-9435-824dbe994ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:21.743442 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b1c79-c46a-45b6-9435-824dbe994ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:21.811748 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-629b1c79-c46a-45b6-9435-824dbe994ad2 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:21.812887 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-629b1c79-c46a-45b6-9435-824dbe994ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:21.835228 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:21.917566 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 06:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:22.228166 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3124000-b3e8-4ad3-b817-02829e4d328c None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:22.229796 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b3124000-b3e8-4ad3-b817-02829e4d328c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:22.349546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3124000-b3e8-4ad3-b817-02829e4d328c None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:22.369807 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:22.425867 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172521c-b471-4c04-ac30-9ada3032955b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:22.426638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172521c-b471-4c04-ac30-9ada3032955b service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:22.426638 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172521c-b471-4c04-ac30-9ada3032955b service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:22.427177 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172521c-b471-4c04-ac30-9ada3032955b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:22.428759 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1172521c-b471-4c04-ac30-9ada3032955b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:22.544233 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172521c-b471-4c04-ac30-9ada3032955b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:22.545696 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172521c-b471-4c04-ac30-9ada3032955b service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:22.564580 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 721/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:24.646229 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9816173-9713-4c2d-b9a9-eba098ae9bff None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:24.646870 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9816173-9713-4c2d-b9a9-eba098ae9bff None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:24.647050 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9816173-9713-4c2d-b9a9-eba098ae9bff None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:24.648444 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 06:07:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:24.654367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f29c53-bd72-45a7-a8c1-7a28113e792e None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:24.654886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f29c53-bd72-45a7-a8c1-7a28113e792e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:24.654972 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f29c53-bd72-45a7-a8c1-7a28113e792e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:24.721627 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:24.725633 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a5f29c53-bd72-45a7-a8c1-7a28113e792e None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:24.727461 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a5f29c53-bd72-45a7-a8c1-7a28113e792e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:24.828049 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 06:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:25.113820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9816173-9713-4c2d-b9a9-eba098ae9bff None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:25.115735 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c9816173-9713-4c2d-b9a9-eba098ae9bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:25.224314 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9816173-9713-4c2d-b9a9-eba098ae9bff None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:25.278849 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:25.306459 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a095c92-9d05-4026-b019-a6d524909b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:25.307098 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a095c92-9d05-4026-b019-a6d524909b87 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:25.307377 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a095c92-9d05-4026-b019-a6d524909b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:25.307653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a095c92-9d05-4026-b019-a6d524909b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:25.309295 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-6a095c92-9d05-4026-b019-a6d524909b87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:25.436920 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a095c92-9d05-4026-b019-a6d524909b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:25.438410 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a095c92-9d05-4026-b019-a6d524909b87 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:25.458766 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 733/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:27.522283 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff4490-d5d9-4238-a3c7-2b75612b0cd2 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:27.522948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff4490-d5d9-4238-a3c7-2b75612b0cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:27.522948 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff4490-d5d9-4238-a3c7-2b75612b0cd2 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:27.524204 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 06:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:27.529589 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcfc7d-7fe6-42d9-ba0c-727ec77a7587 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:27.530056 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcfc7d-7fe6-42d9-ba0c-727ec77a7587 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:27.530160 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcfc7d-7fe6-42d9-ba0c-727ec77a7587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:27.602194 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-2fdcfc7d-7fe6-42d9-ba0c-727ec77a7587 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:27.603637 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-2fdcfc7d-7fe6-42d9-ba0c-727ec77a7587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:27.635014 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:27.705581 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 06:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:27.994203 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff4490-d5d9-4238-a3c7-2b75612b0cd2 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:27.995973 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-18ff4490-d5d9-4238-a3c7-2b75612b0cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:28.107179 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff4490-d5d9-4238-a3c7-2b75612b0cd2 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:28.136881 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:28.182898 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffb3c0-2574-463d-9fb5-8baff20512e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:28.183512 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffb3c0-2574-463d-9fb5-8baff20512e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:28.183787 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffb3c0-2574-463d-9fb5-8baff20512e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:28.184055 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffb3c0-2574-463d-9fb5-8baff20512e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:28.185651 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-95ffb3c0-2574-463d-9fb5-8baff20512e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:28.309457 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffb3c0-2574-463d-9fb5-8baff20512e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:28.311202 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffb3c0-2574-463d-9fb5-8baff20512e8 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:28.330711 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 724/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:30.445882 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75a3a6-976b-46ca-ad83-a074e2a34d36 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:30.446472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75a3a6-976b-46ca-ad83-a074e2a34d36 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:30.446472 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75a3a6-976b-46ca-ad83-a074e2a34d36 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:30.447995 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 06:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:30.453380 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfc031-91b9-4173-b7b6-4a45c86fcac6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:30.453871 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfc031-91b9-4173-b7b6-4a45c86fcac6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:30.453975 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfc031-91b9-4173-b7b6-4a45c86fcac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:30.533297 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-04cfc031-91b9-4173-b7b6-4a45c86fcac6 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:30.534546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-04cfc031-91b9-4173-b7b6-4a45c86fcac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:30.543815 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:30.637523 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 06:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:30.930176 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75a3a6-976b-46ca-ad83-a074e2a34d36 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:30.931764 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-cd75a3a6-976b-46ca-ad83-a074e2a34d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:31.047422 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75a3a6-976b-46ca-ad83-a074e2a34d36 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:31.108306 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:31.128240 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14fc32-9e13-4600-9ac8-7366c06073bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:31.129349 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14fc32-9e13-4600-9ac8-7366c06073bc service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:31.129736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14fc32-9e13-4600-9ac8-7366c06073bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:31.130017 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14fc32-9e13-4600-9ac8-7366c06073bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:31.131662 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-9b14fc32-9e13-4600-9ac8-7366c06073bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:31.245985 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14fc32-9e13-4600-9ac8-7366c06073bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:31.247375 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14fc32-9e13-4600-9ac8-7366c06073bc service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:31.266153 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 736/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:33.270199 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3c711-908d-424b-90df-1184c8f0fc84 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:33.270795 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3c711-908d-424b-90df-1184c8f0fc84 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:33.271053 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3c711-908d-424b-90df-1184c8f0fc84 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:33.272265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 06:07:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:33.277769 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d46f88-8501-4371-9432-5932e68bf4fb None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:33.278248 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d46f88-8501-4371-9432-5932e68bf4fb None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:33.278360 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d46f88-8501-4371-9432-5932e68bf4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:33.352589 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-51d46f88-8501-4371-9432-5932e68bf4fb None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:33.353813 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-51d46f88-8501-4371-9432-5932e68bf4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:33.396528 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:33.460859 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 06:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:33.746598 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3c711-908d-424b-90df-1184c8f0fc84 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:33.748141 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-02e3c711-908d-424b-90df-1184c8f0fc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:33.859655 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3c711-908d-424b-90df-1184c8f0fc84 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:33.899146 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:33.945344 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e2892-76b2-410f-98c8-dfef538f8968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:33.945957 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e2892-76b2-410f-98c8-dfef538f8968 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:33.946236 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e2892-76b2-410f-98c8-dfef538f8968 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:33.946551 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e2892-76b2-410f-98c8-dfef538f8968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:33.950413 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-1a2e2892-76b2-410f-98c8-dfef538f8968 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:34.069795 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e2892-76b2-410f-98c8-dfef538f8968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:34.071272 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e2892-76b2-410f-98c8-dfef538f8968 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:34.093753 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 727/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:36.029310 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c545f-d50f-4819-8c39-537458a517eb None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:36.029868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c545f-d50f-4819-8c39-537458a517eb None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:36.029868 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c545f-d50f-4819-8c39-537458a517eb None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:36.031229 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 06:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:36.037539 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc6ecb-012a-4dda-ac84-85047e3d619a None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:36.038038 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc6ecb-012a-4dda-ac84-85047e3d619a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:36.038153 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc6ecb-012a-4dda-ac84-85047e3d619a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:36.108251 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-82bc6ecb-012a-4dda-ac84-85047e3d619a None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:36.109482 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-82bc6ecb-012a-4dda-ac84-85047e3d619a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:36.128551 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:36.220313 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 06:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:36.502121 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c545f-d50f-4819-8c39-537458a517eb None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:36.503709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-0a0c545f-d50f-4819-8c39-537458a517eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:36.613899 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c545f-d50f-4819-8c39-537458a517eb None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:36.686104 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:36.694507 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1f7dd-7af2-4c44-82f7-044f1ef35d7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:36.695141 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1f7dd-7af2-4c44-82f7-044f1ef35d7e service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:36.695417 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1f7dd-7af2-4c44-82f7-044f1ef35d7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:36.695683 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1f7dd-7af2-4c44-82f7-044f1ef35d7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:36.697304 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-26a1f7dd-7af2-4c44-82f7-044f1ef35d7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:36.818803 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1f7dd-7af2-4c44-82f7-044f1ef35d7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:36.820271 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1f7dd-7af2-4c44-82f7-044f1ef35d7e service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:36.838845 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 739/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:38.802259 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1e366-b524-4ee6-bf6a-bcac29ecca6a None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:38.802888 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1e366-b524-4ee6-bf6a-bcac29ecca6a None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:38.802888 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1e366-b524-4ee6-bf6a-bcac29ecca6a None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:38.804184 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 06:07:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:07:38.809904 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48c6a2-e9ec-4aa2-8ea0-acd6f5b287b0 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:38.810365 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48c6a2-e9ec-4aa2-8ea0-acd6f5b287b0 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:38.810474 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48c6a2-e9ec-4aa2-8ea0-acd6f5b287b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:38.878582 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-cc48c6a2-e9ec-4aa2-8ea0-acd6f5b287b0 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:07:38.880463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-cc48c6a2-e9ec-4aa2-8ea0-acd6f5b287b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:38.935167 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:38.990842 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 06:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:07:39.271393 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1e366-b524-4ee6-bf6a-bcac29ecca6a None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:07:39.272950 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-fcb1e366-b524-4ee6-bf6a-bcac29ecca6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:39.384838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1e366-b524-4ee6-bf6a-bcac29ecca6a None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:07:39.437635 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:39.465479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dcf05-0ef3-4f52-a693-d59504987397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:07:39.466086 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dcf05-0ef3-4f52-a693-d59504987397 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:07:39.466387 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dcf05-0ef3-4f52-a693-d59504987397 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:07:39.466735 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dcf05-0ef3-4f52-a693-d59504987397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:07:39.468769 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-786dcf05-0ef3-4f52-a693-d59504987397 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:07:39.590539 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786dcf05-0ef3-4f52-a693-d59504987397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:07:39.592024 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786dcf05-0ef3-4f52-a693-d59504987397 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:07:39.613493 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 730/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:07:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:07:48.989220 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=22 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:07:49.610681 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=66 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:08:00.947071 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ce0a1-4055-4865-b086-46437a6f3f95 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:00.947902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ce0a1-4055-4865-b086-46437a6f3f95 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:00.947902 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ce0a1-4055-4865-b086-46437a6f3f95 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:00.949145 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 741/1471] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:08:00.954381 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c623d-588b-4eae-88e4-8f2cd400a195 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:00.954897 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c623d-588b-4eae-88e4-8f2cd400a195 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:00.955023 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c623d-588b-4eae-88e4-8f2cd400a195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:01.026026 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-e17c623d-588b-4eae-88e4-8f2cd400a195 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:08:01.027316 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-e17c623d-588b-4eae-88e4-8f2cd400a195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:01.126653 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 731/1472] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:08:01.423235 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559617f-861f-4b16-af07-6fc8f253d111 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:01.423522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559617f-861f-4b16-af07-6fc8f253d111 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:01.423820 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559617f-861f-4b16-af07-6fc8f253d111 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:01.425213 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 742/1473] 10.222.0.20 () {54 vars in 967 bytes} [Tue Aug 23 06:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:08:01.430196 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9379f44-54a9-4cfd-8def-48895df3117e None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:01.430479 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9379f44-54a9-4cfd-8def-48895df3117e None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:01.430781 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9379f44-54a9-4cfd-8def-48895df3117e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:01.506390 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-a9379f44-54a9-4cfd-8def-48895df3117e None None] MFA Rules not processed for user `89621d839d1c44798f691db4d49e1278`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:08:01.507611 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-a9379f44-54a9-4cfd-8def-48895df3117e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:01.528146 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:08:01.594190 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 732/1474] 10.222.0.20 () {58 vars in 1064 bytes} [Tue Aug 23 06:08:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:08:01.597886 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559617f-861f-4b16-af07-6fc8f253d111 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:08:01.600131 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-1559617f-861f-4b16-af07-6fc8f253d111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:01.717752 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559617f-861f-4b16-af07-6fc8f253d111 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:08:01.798893 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042255c-008f-42bf-979b-65460ee1377b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89621d839d1c44798f691db4d49e1278', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:08:01.799513 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042255c-008f-42bf-979b-65460ee1377b service glance] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:01.799791 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042255c-008f-42bf-979b-65460ee1377b service glance] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:01.800090 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042255c-008f-42bf-979b-65460ee1377b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:01.802088 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a042255c-008f-42bf-979b-65460ee1377b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:01.872522 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:08:01.931706 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042255c-008f-42bf-979b-65460ee1377b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:08:01.933078 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042255c-008f-42bf-979b-65460ee1377b service glance] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:08:01.954001 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 743/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:08:04.219207 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae0643-beac-42ce-90a2-d7bbf334d4c8 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:04.219736 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae0643-beac-42ce-90a2-d7bbf334d4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:04.219736 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae0643-beac-42ce-90a2-d7bbf334d4c8 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:04.221045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 733/1476] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:08:04.226971 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e1d02-6a32-4e44-aa74-a54615f592e3 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:04.227452 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e1d02-6a32-4e44-aa74-a54615f592e3 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:04.227562 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e1d02-6a32-4e44-aa74-a54615f592e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:04.295460 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-de0e1d02-6a32-4e44-aa74-a54615f592e3 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:08:04.296618 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-de0e1d02-6a32-4e44-aa74-a54615f592e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:04.521183 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 744/1477] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:08:04.803825 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae0643-beac-42ce-90a2-d7bbf334d4c8 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:08:04.805280 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d3ae0643-beac-42ce-90a2-d7bbf334d4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:05.062987 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae0643-beac-42ce-90a2-d7bbf334d4c8 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:08:05.115685 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:08:05.161518 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c81ff-76b0-4bea-aa5d-3922d7f14979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e3a96fd05742d9a58f3a78e19ef4f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:08:05.162116 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c81ff-76b0-4bea-aa5d-3922d7f14979 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:05.162386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c81ff-76b0-4bea-aa5d-3922d7f14979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:05.162736 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c81ff-76b0-4bea-aa5d-3922d7f14979 admin admin] PATH_INFO: `/v3/projects` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:05.163360 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c81ff-76b0-4bea-aa5d-3922d7f14979 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:08:05.165164 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: WARNING py.warnings [None req-3e9c81ff-76b0-4bea-aa5d-3922d7f14979 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 06:08:05.165164 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: warnings.warn(msg) Aug 23 06:08:05.165164 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: Aug 23 06:08:05.166299 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c81ff-76b0-4bea-aa5d-3922d7f14979 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:08:05.185812 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 734/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Aug 23 06:08:04 2022] GET /identity/v3/projects => generated 2553 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 06:08:07.153625 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02401ae1-f3ac-49b8-9828-0123c213d726 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:07.154182 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02401ae1-f3ac-49b8-9828-0123c213d726 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:07.154294 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02401ae1-f3ac-49b8-9828-0123c213d726 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:07.155716 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 745/1479] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:08:07.161815 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7c32b-86e9-4b6f-ac7b-cceaee5e3ec6 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:07.162274 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7c32b-86e9-4b6f-ac7b-cceaee5e3ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:07.162381 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7c32b-86e9-4b6f-ac7b-cceaee5e3ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:07.230982 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-13d7c32b-86e9-4b6f-ac7b-cceaee5e3ec6 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:08:07.232114 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-13d7c32b-86e9-4b6f-ac7b-cceaee5e3ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:07.336942 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 735/1480] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:08:07.645617 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02401ae1-f3ac-49b8-9828-0123c213d726 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:08:07.647632 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-02401ae1-f3ac-49b8-9828-0123c213d726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:07.658673 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:08:07.798830 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02401ae1-f3ac-49b8-9828-0123c213d726 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:08:07.882221 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:08:07.920508 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49071ead-0484-42b9-9cb8-3de68049746a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91267a049343452eb208205ee3ea27c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:08:07.921822 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49071ead-0484-42b9-9cb8-3de68049746a service nova] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:07.922237 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49071ead-0484-42b9-9cb8-3de68049746a service nova] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:07.922681 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49071ead-0484-42b9-9cb8-3de68049746a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:07.926173 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-49071ead-0484-42b9-9cb8-3de68049746a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:08.027294 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49071ead-0484-42b9-9cb8-3de68049746a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:08:08.028883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49071ead-0484-42b9-9cb8-3de68049746a service nova] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:08:08.047847 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 746/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:08:10.199052 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0022dc2-a681-4689-85ee-b4370eafca90 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:10.199621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0022dc2-a681-4689-85ee-b4370eafca90 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:10.199621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0022dc2-a681-4689-85ee-b4370eafca90 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:10.201003 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 736/1482] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:08:10.207041 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94133c9e-4f12-4b8c-bd87-4c62b5667acf None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:10.207505 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94133c9e-4f12-4b8c-bd87-4c62b5667acf None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:10.207505 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94133c9e-4f12-4b8c-bd87-4c62b5667acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:10.276226 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-94133c9e-4f12-4b8c-bd87-4c62b5667acf None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:08:10.277443 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-94133c9e-4f12-4b8c-bd87-4c62b5667acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:10.327950 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:08:10.387119 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 747/1483] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:08:10.677753 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0022dc2-a681-4689-85ee-b4370eafca90 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:08:10.679369 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-b0022dc2-a681-4689-85ee-b4370eafca90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:10.704161 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:08:10.791414 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0022dc2-a681-4689-85ee-b4370eafca90 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:08:10.870667 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e497c-48b5-4275-a71c-2c7aac9cb4d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:08:10.871279 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e497c-48b5-4275-a71c-2c7aac9cb4d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:10.871546 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e497c-48b5-4275-a71c-2c7aac9cb4d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:10.871813 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e497c-48b5-4275-a71c-2c7aac9cb4d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:10.873401 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-d69e497c-48b5-4275-a71c-2c7aac9cb4d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:10.958652 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:08:11.001362 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e497c-48b5-4275-a71c-2c7aac9cb4d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:08:11.003640 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e497c-48b5-4275-a71c-2c7aac9cb4d1 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:08:11.023212 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 737/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 06:08:13.109518 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdee5fc-6110-4056-ba7d-e950979daee6 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:13.110045 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdee5fc-6110-4056-ba7d-e950979daee6 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:13.110170 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdee5fc-6110-4056-ba7d-e950979daee6 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:13.111463 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 748/1485] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:08:13.118258 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598973ce-7735-417d-ac3f-c0e02b5b83a9 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:13.118752 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598973ce-7735-417d-ac3f-c0e02b5b83a9 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:13.118849 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598973ce-7735-417d-ac3f-c0e02b5b83a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:13.199963 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-598973ce-7735-417d-ac3f-c0e02b5b83a9 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:08:13.201210 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-598973ce-7735-417d-ac3f-c0e02b5b83a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:13.302532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 738/1486] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:08:13.591653 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdee5fc-6110-4056-ba7d-e950979daee6 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:08:13.593198 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-3cdee5fc-6110-4056-ba7d-e950979daee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:13.701358 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdee5fc-6110-4056-ba7d-e950979daee6 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:08:13.777306 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:08:13.779285 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873556e5-2da4-4d2e-a19a-384d2ae0c29d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:08:13.780019 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873556e5-2da4-4d2e-a19a-384d2ae0c29d service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:13.780343 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873556e5-2da4-4d2e-a19a-384d2ae0c29d service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:13.780859 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873556e5-2da4-4d2e-a19a-384d2ae0c29d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:13.783024 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-873556e5-2da4-4d2e-a19a-384d2ae0c29d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:13.905871 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873556e5-2da4-4d2e-a19a-384d2ae0c29d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:08:13.908028 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873556e5-2da4-4d2e-a19a-384d2ae0c29d service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:08:13.927991 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 749/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:08:16.053503 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47402d96-e912-4ba3-9683-7e108d1750a9 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:16.054083 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47402d96-e912-4ba3-9683-7e108d1750a9 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:16.054083 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47402d96-e912-4ba3-9683-7e108d1750a9 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:16.055442 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 739/1488] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:08:16.061033 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d08e93-9483-4a28-9089-7d7e05842a2d None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:16.061535 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d08e93-9483-4a28-9089-7d7e05842a2d None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:16.061644 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d08e93-9483-4a28-9089-7d7e05842a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:16.131340 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-c4d08e93-9483-4a28-9089-7d7e05842a2d None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:08:16.132554 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-c4d08e93-9483-4a28-9089-7d7e05842a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:16.191026 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:08:16.241501 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 750/1489] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:08:16.533265 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47402d96-e912-4ba3-9683-7e108d1750a9 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:08:16.535568 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-47402d96-e912-4ba3-9683-7e108d1750a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:16.583379 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:08:16.657388 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47402d96-e912-4ba3-9683-7e108d1750a9 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:08:16.736200 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b992b7c-e1ae-4d81-9d4e-dffb12aab8a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:08:16.736816 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b992b7c-e1ae-4d81-9d4e-dffb12aab8a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:16.737277 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b992b7c-e1ae-4d81-9d4e-dffb12aab8a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:16.737686 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b992b7c-e1ae-4d81-9d4e-dffb12aab8a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:16.739420 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-3b992b7c-e1ae-4d81-9d4e-dffb12aab8a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:16.828876 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:08:16.868621 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b992b7c-e1ae-4d81-9d4e-dffb12aab8a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:08:16.868870 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b992b7c-e1ae-4d81-9d4e-dffb12aab8a0 service neutron] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:08:16.892203 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 740/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:08:19.036982 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c374e9c-aaea-4978-9e7e-4ec12e9b44f1 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:19.037711 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c374e9c-aaea-4978-9e7e-4ec12e9b44f1 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:19.037711 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c374e9c-aaea-4978-9e7e-4ec12e9b44f1 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:19.038992 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 751/1491] 10.222.0.20 () {54 vars in 927 bytes} [Tue Aug 23 06:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:08:19.044875 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebc04c-5c56-4c1c-8267-c121621f24af None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:19.045541 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebc04c-5c56-4c1c-8267-c121621f24af None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:19.045541 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebc04c-5c56-4c1c-8267-c121621f24af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:19.128047 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-2eebc04c-5c56-4c1c-8267-c121621f24af None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:08:19.129304 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-2eebc04c-5c56-4c1c-8267-c121621f24af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:19.229926 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 741/1492] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Aug 23 06:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:08:19.518180 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c374e9c-aaea-4978-9e7e-4ec12e9b44f1 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:08:19.519906 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5c374e9c-aaea-4978-9e7e-4ec12e9b44f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:19.623898 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c374e9c-aaea-4978-9e7e-4ec12e9b44f1 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:08:19.700915 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9180580-4227-42e4-ac51-876a4fab9646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:08:19.701514 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9180580-4227-42e4-ac51-876a4fab9646 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:08:19.701787 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9180580-4227-42e4-ac51-876a4fab9646 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:08:19.702047 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9180580-4227-42e4-ac51-876a4fab9646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:08:19.703645 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-a9180580-4227-42e4-ac51-876a4fab9646 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:08:19.713122 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:08:19.819832 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9180580-4227-42e4-ac51-876a4fab9646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:08:19.821209 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9180580-4227-42e4-ac51-876a4fab9646 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:08:19.839483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 752/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:08:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:08:29.228144 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=82 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:08:29.837224 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=50 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:09:51.200288 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca7eb5-f6ab-41d6-9971-db191350c1a2 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:51.200885 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca7eb5-f6ab-41d6-9971-db191350c1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:51.200885 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca7eb5-f6ab-41d6-9971-db191350c1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:51.279994 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-3aca7eb5-f6ab-41d6-9971-db191350c1a2 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:09:51.281206 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-3aca7eb5-f6ab-41d6-9971-db191350c1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:51.383178 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 742/1494] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 23 06:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:09:51.391949 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c2e2c-3682-4cf7-a3ec-ee1d9e249d77 None None] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:51.392574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c2e2c-3682-4cf7-a3ec-ee1d9e249d77 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:51.392574 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c2e2c-3682-4cf7-a3ec-ee1d9e249d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:51.476867 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.auth.core [None req-618c2e2c-3682-4cf7-a3ec-ee1d9e249d77 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81747) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:09:51.478255 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-618c2e2c-3682-4cf7-a3ec-ee1d9e249d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:51.585608 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 753/1495] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 23 06:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:09:51.590367 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca7eb5-f6ab-41d6-9971-db191350c1a2 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:09:51.591653 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-3aca7eb5-f6ab-41d6-9971-db191350c1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:51.604189 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca7eb5-f6ab-41d6-9971-db191350c1a2 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:09:51.679989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:09:51.715838 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de5c25-fea6-4ec5-aa7b-fe09f80f5af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e3a96fd05742d9a58f3a78e19ef4f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:09:51.716437 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de5c25-fea6-4ec5-aa7b-fe09f80f5af4 admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:51.716705 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de5c25-fea6-4ec5-aa7b-fe09f80f5af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:51.716969 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de5c25-fea6-4ec5-aa7b-fe09f80f5af4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:51.717610 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de5c25-fea6-4ec5-aa7b-fe09f80f5af4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81746) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 06:09:51.717902 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de5c25-fea6-4ec5-aa7b-fe09f80f5af4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:09:51.719111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: WARNING py.warnings [None req-67de5c25-fea6-4ec5-aa7b-fe09f80f5af4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 06:09:51.719111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: warnings.warn(msg) Aug 23 06:09:51.719111 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: Aug 23 06:09:51.720077 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de5c25-fea6-4ec5-aa7b-fe09f80f5af4 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:09:51.731679 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 743/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Aug 23 06:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 06:09:51.740890 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c2e2c-3682-4cf7-a3ec-ee1d9e249d77 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:09:51.751728 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c2e2c-3682-4cf7-a3ec-ee1d9e249d77 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:09:51.818251 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:09:51.854023 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefb23b-3a30-42c3-99b1-4e00076ca70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e3a96fd05742d9a58f3a78e19ef4f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:09:51.854541 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefb23b-3a30-42c3-99b1-4e00076ca70d admin admin] REQUEST_METHOD: `POST` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:51.854821 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefb23b-3a30-42c3-99b1-4e00076ca70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:51.855089 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefb23b-3a30-42c3-99b1-4e00076ca70d admin admin] PATH_INFO: `/v3/projects` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:51.855713 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefb23b-3a30-42c3-99b1-4e00076ca70d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:09:51.857468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: WARNING py.warnings [None req-5fefb23b-3a30-42c3-99b1-4e00076ca70d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 06:09:51.857468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: warnings.warn(msg) Aug 23 06:09:51.857468 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: Aug 23 06:09:51.858709 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefb23b-3a30-42c3-99b1-4e00076ca70d admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:09:51.921328 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 754/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Aug 23 06:09:51 2022] POST /identity/v3/projects => generated 379 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:09:51.926302 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de5c25-fea6-4ec5-aa7b-fe09f80f5af4 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:09:51.927852 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-67de5c25-fea6-4ec5-aa7b-fe09f80f5af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:51.971385 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de5c25-fea6-4ec5-aa7b-fe09f80f5af4 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:09:52.072886 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373f4b8-eb17-4ce7-acaa-d7a48ffe2778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e3a96fd05742d9a58f3a78e19ef4f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:09:52.073499 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373f4b8-eb17-4ce7-acaa-d7a48ffe2778 admin admin] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:52.073908 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373f4b8-eb17-4ce7-acaa-d7a48ffe2778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:52.074178 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373f4b8-eb17-4ce7-acaa-d7a48ffe2778 admin admin] PATH_INFO: `/v3/users` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:52.074844 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373f4b8-eb17-4ce7-acaa-d7a48ffe2778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:09:52.075873 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: WARNING py.warnings [None req-5373f4b8-eb17-4ce7-acaa-d7a48ffe2778 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 06:09:52.075873 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: warnings.warn(msg) Aug 23 06:09:52.075873 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: Aug 23 06:09:52.077175 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373f4b8-eb17-4ce7-acaa-d7a48ffe2778 admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:09:52.127447 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 744/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Aug 23 06:09:51 2022] POST /identity/v3/users => generated 290 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 06:09:52.128872 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefb23b-3a30-42c3-99b1-4e00076ca70d admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:09:52.130417 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-5fefb23b-3a30-42c3-99b1-4e00076ca70d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:52.195879 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefb23b-3a30-42c3-99b1-4e00076ca70d admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:09:52.292726 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:09:52.305334 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556982e-8f1a-4747-92bc-dfa27926ddbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e3a96fd05742d9a58f3a78e19ef4f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:09:52.306285 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556982e-8f1a-4747-92bc-dfa27926ddbe admin admin] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:52.306285 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556982e-8f1a-4747-92bc-dfa27926ddbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:52.306483 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556982e-8f1a-4747-92bc-dfa27926ddbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:52.307180 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556982e-8f1a-4747-92bc-dfa27926ddbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:09:52.308329 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: WARNING py.warnings [None req-b556982e-8f1a-4747-92bc-dfa27926ddbe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 06:09:52.308329 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: warnings.warn(msg) Aug 23 06:09:52.308329 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: Aug 23 06:09:52.309212 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556982e-8f1a-4747-92bc-dfa27926ddbe admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:09:52.322370 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 755/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 23 06:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 06:09:52.326476 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373f4b8-eb17-4ce7-acaa-d7a48ffe2778 admin admin] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:09:52.328142 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-5373f4b8-eb17-4ce7-acaa-d7a48ffe2778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:52.340797 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373f4b8-eb17-4ce7-acaa-d7a48ffe2778 admin admin] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:09:52.366583 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:09:52.439915 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060731c2-1c73-436d-a6d5-7bf90a4dba6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e3a96fd05742d9a58f3a78e19ef4f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:09:52.440437 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060731c2-1c73-436d-a6d5-7bf90a4dba6d admin admin] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:52.440763 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060731c2-1c73-436d-a6d5-7bf90a4dba6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:52.440989 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060731c2-1c73-436d-a6d5-7bf90a4dba6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:52.441631 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060731c2-1c73-436d-a6d5-7bf90a4dba6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:09:52.442780 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: WARNING py.warnings [None req-060731c2-1c73-436d-a6d5-7bf90a4dba6d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 06:09:52.442780 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: warnings.warn(msg) Aug 23 06:09:52.442780 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: Aug 23 06:09:52.443618 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060731c2-1c73-436d-a6d5-7bf90a4dba6d admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:09:52.457631 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 745/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Aug 23 06:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 06:09:52.461844 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556982e-8f1a-4747-92bc-dfa27926ddbe admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:09:52.463401 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-b556982e-8f1a-4747-92bc-dfa27926ddbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:52.498506 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556982e-8f1a-4747-92bc-dfa27926ddbe admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:09:52.601355 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d338c-8238-4bd2-8837-68a45cbe58b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e3a96fd05742d9a58f3a78e19ef4f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:09:52.602010 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d338c-8238-4bd2-8837-68a45cbe58b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:52.602288 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d338c-8238-4bd2-8837-68a45cbe58b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:52.602585 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d338c-8238-4bd2-8837-68a45cbe58b2 admin admin] PATH_INFO: `/v3/projects/c2a88106f7f047c4930018f5914873e4/users/18eaa9fbd96d47c1b3aeb49058916980/roles/a2f35dad17e94553b63d07f8b8d8f1a4` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:52.624664 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d338c-8238-4bd2-8837-68a45cbe58b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2a88106f7f047c4930018f5914873e4, user_id=18eaa9fbd96d47c1b3aeb49058916980, role_id=a2f35dad17e94553b63d07f8b8d8f1a4)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:09:52.626335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: WARNING py.warnings [None req-bf6d338c-8238-4bd2-8837-68a45cbe58b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 06:09:52.626335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: warnings.warn(msg) Aug 23 06:09:52.626335 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: Aug 23 06:09:52.628588 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d338c-8238-4bd2-8837-68a45cbe58b2 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:09:52.657614 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 756/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Aug 23 06:09:52 2022] PUT /identity/v3/projects/c2a88106f7f047c4930018f5914873e4/users/18eaa9fbd96d47c1b3aeb49058916980/roles/a2f35dad17e94553b63d07f8b8d8f1a4 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:09:52.670234 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc65ea-ef52-4e4c-b474-a12ab02a117d None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:52.670540 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc65ea-ef52-4e4c-b474-a12ab02a117d None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:52.670816 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc65ea-ef52-4e4c-b474-a12ab02a117d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:52.729662 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-89dc65ea-ef52-4e4c-b474-a12ab02a117d None None] MFA Rules not processed for user `18eaa9fbd96d47c1b3aeb49058916980`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:09:52.730821 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-89dc65ea-ef52-4e4c-b474-a12ab02a117d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:52.814234 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 746/1502] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 23 06:09:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:09:52.827751 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19da04-d7ae-4dad-91a8-f900335a6762 None None] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:52.828134 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19da04-d7ae-4dad-91a8-f900335a6762 None None] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:52.829203 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19da04-d7ae-4dad-91a8-f900335a6762 None None] PATH_INFO: `/` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:52.830720 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 757/1503] 10.222.0.20 () {54 vars in 964 bytes} [Tue Aug 23 06:09:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:09:52.836087 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf6412-a3ed-4faf-8d06-9b31ce119cc0 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:52.836372 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf6412-a3ed-4faf-8d06-9b31ce119cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:52.836649 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf6412-a3ed-4faf-8d06-9b31ce119cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:52.868603 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:09:52.917536 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-9edf6412-a3ed-4faf-8d06-9b31ce119cc0 None None] MFA Rules not processed for user `c741b92684794c338d63913774ab841a`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:09:52.918791 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-9edf6412-a3ed-4faf-8d06-9b31ce119cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:53.003718 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 747/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Tue Aug 23 06:09:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 06:09:53.011323 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19da04-d7ae-4dad-91a8-f900335a6762 None None] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:09:53.012838 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-cb19da04-d7ae-4dad-91a8-f900335a6762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:53.080042 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:09:53.144294 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19da04-d7ae-4dad-91a8-f900335a6762 None None] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:09:53.218149 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cf8de-fc43-4e40-872d-c9282667e83f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c741b92684794c338d63913774ab841a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:09:53.218782 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cf8de-fc43-4e40-872d-c9282667e83f service cinder] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:53.219238 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cf8de-fc43-4e40-872d-c9282667e83f service cinder] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:53.219514 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cf8de-fc43-4e40-872d-c9282667e83f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:53.221108 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-462cf8de-fc43-4e40-872d-c9282667e83f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:53.325909 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cf8de-fc43-4e40-872d-c9282667e83f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:09:53.327323 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cf8de-fc43-4e40-872d-c9282667e83f service cinder] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:09:53.331083 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:09:53.346234 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 758/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:09:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 06:09:53.392884 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048b3be-0f4c-487d-8816-4aa31dc2c503 None None] REQUEST_METHOD: `GET` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:53.393369 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048b3be-0f4c-487d-8816-4aa31dc2c503 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:53.393484 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048b3be-0f4c-487d-8816-4aa31dc2c503 None None] PATH_INFO: `/` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:53.394725 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 748/1506] 10.222.0.20 () {52 vars in 830 bytes} [Tue Aug 23 06:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 06:09:53.398542 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cf8de-fc43-4e40-872d-c9282667e83f service cinder] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:09:53.399853 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-462cf8de-fc43-4e40-872d-c9282667e83f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:53.506736 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cf8de-fc43-4e40-872d-c9282667e83f service cinder] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:09:53.604671 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb408da-5e30-4026-b996-2f1541aceeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e3a96fd05742d9a58f3a78e19ef4f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:09:53.605216 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb408da-5e30-4026-b996-2f1541aceeb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:53.605491 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb408da-5e30-4026-b996-2f1541aceeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:53.605759 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb408da-5e30-4026-b996-2f1541aceeb1 admin admin] PATH_INFO: `/v3/users/18eaa9fbd96d47c1b3aeb49058916980` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:53.619341 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb408da-5e30-4026-b996-2f1541aceeb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18eaa9fbd96d47c1b3aeb49058916980)` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:09:53.620380 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: WARNING py.warnings [None req-7fb408da-5e30-4026-b996-2f1541aceeb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 06:09:53.620380 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: warnings.warn(msg) Aug 23 06:09:53.620380 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: Aug 23 06:09:53.621603 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb408da-5e30-4026-b996-2f1541aceeb1 admin admin] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:09:53.632179 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=100 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:09:53.696628 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7fb408da-5e30-4026-b996-2f1541aceeb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18eaa9fbd96d47c1b3aeb49058916980'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 06:09:53.720441 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7fb408da-5e30-4026-b996-2f1541aceeb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18eaa9fbd96d47c1b3aeb49058916980'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 06:09:53.721029 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7fb408da-5e30-4026-b996-2f1541aceeb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18eaa9fbd96d47c1b3aeb49058916980'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 06:09:53.737411 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7fb408da-5e30-4026-b996-2f1541aceeb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18eaa9fbd96d47c1b3aeb49058916980'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 06:09:53.737883 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.notifications [None req-7fb408da-5e30-4026-b996-2f1541aceeb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18eaa9fbd96d47c1b3aeb49058916980'} {{(pid=81747) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 06:09:53.753558 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 759/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Aug 23 06:09:53 2022] DELETE /identity/v3/users/18eaa9fbd96d47c1b3aeb49058916980 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:09:53.759570 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeabbd-c488-4d49-a8f8-8c1691cfb839 None None] REQUEST_METHOD: `POST` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:53.760469 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeabbd-c488-4d49-a8f8-8c1691cfb839 None None] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:53.761386 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeabbd-c488-4d49-a8f8-8c1691cfb839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:53.826098 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.auth.core [None req-54eeabbd-c488-4d49-a8f8-8c1691cfb839 None None] MFA Rules not processed for user `c922803daa44438cbb8185e3898e3dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81746) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 06:09:53.827336 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-54eeabbd-c488-4d49-a8f8-8c1691cfb839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:53.912128 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=100 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:09:53.926709 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 749/1508] 10.222.0.20 () {58 vars in 958 bytes} [Tue Aug 23 06:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 06:09:53.936132 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb408da-5e30-4026-b996-2f1541aceeb1 admin admin] Authenticating user token {{(pid=81747) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:09:53.938284 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-7fb408da-5e30-4026-b996-2f1541aceeb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:54.058157 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb408da-5e30-4026-b996-2f1541aceeb1 admin admin] Validating token access rules against request {{(pid=81747) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:09:54.125461 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:09:54.148132 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6d548-342c-4749-bae5-893a4ef0eb65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dae9eff3bbf04496af2300a24c630eb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0b43dead1664475b9dd52113e08d7ed', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81747) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:09:54.148840 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6d548-342c-4749-bae5-893a4ef0eb65 service neutron] REQUEST_METHOD: `GET` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:54.149114 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6d548-342c-4749-bae5-893a4ef0eb65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:54.149380 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6d548-342c-4749-bae5-893a4ef0eb65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81747) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:54.151067 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.fernet_utils [None req-dac6d548-342c-4749-bae5-893a4ef0eb65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81747) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:54.263497 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6d548-342c-4749-bae5-893a4ef0eb65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:09:54.264932 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6d548-342c-4749-bae5-893a4ef0eb65 service neutron] RBAC: Authorization granted {{(pid=81747) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:09:54.284857 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: [pid: 81747|app: 0|req: 760/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Aug 23 06:09:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 06:09:54.631660 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeabbd-c488-4d49-a8f8-8c1691cfb839 None None] Authenticating user token {{(pid=81746) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 06:09:54.633124 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.fernet_utils [None req-54eeabbd-c488-4d49-a8f8-8c1691cfb839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81746) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 06:09:54.644964 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeabbd-c488-4d49-a8f8-8c1691cfb839 None None] Validating token access rules against request {{(pid=81746) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 06:09:54.759742 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c922803daa44438cbb8185e3898e3dc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1e3a96fd05742d9a58f3a78e19ef4f6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81746) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 06:09:54.760532 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 06:09:54.760879 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 06:09:54.761216 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c admin admin] PATH_INFO: `/v3/projects/c2a88106f7f047c4930018f5914873e4` {{(pid=81746) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 06:09:54.774576 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2a88106f7f047c4930018f5914873e4)` {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 06:09:54.775645 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: WARNING py.warnings [None req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 06:09:54.775645 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: warnings.warn(msg) Aug 23 06:09:54.775645 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: Aug 23 06:09:54.777045 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c admin admin] RBAC: Authorization granted {{(pid=81746) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 06:09:54.851665 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2a88106f7f047c4930018f5914873e4', 'request_id': 'req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 06:09:54.869146 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2a88106f7f047c4930018f5914873e4', 'request_id': 'req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 06:09:54.877574 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2a88106f7f047c4930018f5914873e4', 'request_id': 'req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c'} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 06:09:54.907392 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c admin admin] The token cache is being invalidate because project c2a88106f7f047c4930018f5914873e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81746) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 06:09:54.907719 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 06:09:54.908091 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG keystone.notifications [None req-637a0604-aeb4-4fcb-9c58-5e5d9a89e25c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81746) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 06:09:54.909524 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: [pid: 81746|app: 0|req: 750/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Aug 23 06:09:54 2022] DELETE /identity/v3/projects/c2a88106f7f047c4930018f5914873e4 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 06:10:04.282793 dv-cinder-854070-2-iscsi devstack@keystone.service[81747]: DEBUG dbcounter [-] [81747] Writing DB stats keystone:SELECT=62 {{(pid=81747) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 06:10:04.904623 dv-cinder-854070-2-iscsi devstack@keystone.service[81746]: DEBUG dbcounter [-] [81746] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81746) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}